Re: [Duplicity-talk] Full backups fail but incremental backups succeed

[Peter Schuller]

> monthly full backup has never completed successfully. The full backup  
> fails on different files each month, but always fails with a series of  
> messages like:
> 
> Running 'scp  /mnt/003/duplicity-KefoOq-tempdir/mktemp-PDAbBq-193 
> address@hidden 
> :001/duplicity-full. 
> 2008-08-01T03:00:02+10:00.vol192.difftar.gpg' (attempt #1)
> Stalled for too long, aborted copy
> Running 'scp  /mnt/003/duplicity-KefoOq-tempdir/mktemp-PDAbBq-193 
> address@hidden 
> :001/duplicity-full. 
> 2008-08-01T03:00:02+10:00.vol192.difftar.gpg' (attempt #2)
> Timeout waiting for response
> Running 'scp  /mnt/003/duplicity-KefoOq-tempdir/mktemp-PDAbBq-193 
> address@hidden 
> :001/duplicity-full. 
> 2008-08-01T03:00:02+10:00.vol192.difftar.gpg' (attempt #3)
> Timeout waiting for response
> Running 'scp  /mnt/003/duplicity-KefoOq-tempdir/mktemp-PDAbBq-193 
> address@hidden 
> :001/duplicity-full. 
> 2008-08-01T03:00:02+10:00.vol192.difftar.gpg' (attempt #4)
> Timeout waiting for response
> Running 'scp  /mnt/003/duplicity-KefoOq-tempdir/mktemp-PDAbBq-193 
> address@hidden 
> :001/duplicity-full. 
> 2008-08-01T03:00:02+10:00.vol192.difftar.gpg' (attempt #5)
> Timeout waiting for response
> 
> 
> Any ideas? This happens every month and on a different file each time  
> so the chances of it being a recurrent freak network connectivity  
> issue seem low.

Freak issue may be unlikely, but it does seem to be a systematic
connectivity issue. 

"Timeout waiting for response" refers to a timeout waiting for 'scp'
to print something. 'Stalled for too long' indicates 'scp' actually
said 'stalled', and then was silent for a significant period.

First of all you can try:

  --timeout=300

Which will increase the timeout duplicity uses when waiting for output
from scp. This could cause backups to complete successfully, surviging
temporary stalls. The default is 30 seconds.

However, to have a stronger indication whether it is a duplicity
problem or an ssh/networking/remote end problem, I suggest running
with:

  --timeout=300
  --ssh-options='-oServerAliveInterval=5 -o ServerAliveCountMax=6'

This should cause ssh itself to fail and return an error to duplicity,
if it actually suffers a communication timeout with the remote end
(and it should do this after about 30 seconds, while duplicity should
be waiting up to 300 seconds).

This may not be fool proof depending on what the logic looks like in
scp/ssh that decides when to print status updates on the console. But
if the problem is a clear cut connectivity issue, rather than some
weird combination of problems, hopefully this should trigger.

-- 
/ Peter Schuller

PGP userID: 0xE9758B7D or 'Peter Schuller <address@hidden>'
Key retrieval: Send an E-Mail to address@hidden
E-Mail: address@hidden Web: http://www.scode.org

pgpf5dQnPMhXM.pgp
Description: PGP signature