Re: [Duplicity-talk] Duplicity hanging after errors

[edgar . soldin]

On 25.12.2014 01:40, E.B. wrote:
>>> Hi,
>>> I'm running duplicity from a cron script on two ubuntu 12.04 servers
>>> (installed from apt-get, so duplicity version 0.6.25)
> 
> my mistaken, this is a hand-compiled installation of 0.6.25

shouldn't be an issue

>>> Big problem that the error causes duplicity to hang and not exit.
>>> Wondering why that is? If I add "--progress" then it will die (the
>>> trace goes through dummy_backup first where it dies, but
>>> write_multivol just hangs). Is my only solution here to add
>>> "--progress" to my cron script?
>>
>> no, usually you would raise --verbosity=debug (-v9) and run
>> from command line to see if you get the same error, and more
>> details.
> 
> i did that before my first post, the output is from the CLI.
> My question here was precisely that unless this bug becomes
> fixed, is my only way to be certain a cron job doesn't hang
> to use the --progress option?

consider using timeout
  http://serverfault.com/questions/539665/setting-timeout-for-cron-jobs

> right now to be safe, i use --progress with grep -v ETA to
> remove the progress indicators from the cron log. but i'm not
> sure how heavy performance hit is from using --progress

depends on the size of your backup. it does an unnecessary dry run first. also 
it is known to be broken in some scenarios. generally i would advise not to use 
it in the current state, especially not when using duplicity not interactively.
 
>> the error indicates that at least one of the backup's files
>> are corrupted. you seem to use compression but no encryption
>> (right?) as gzip tries to decompress and fails.
> 
> No, I use gpg and asymmetric encryption. Why did you guess I
> don't use GPG? Is it possible I think I'm using GPG but I am
> mistaken?

ok, my bad.. it indicates you use gpg, but that duplicity tarfile should be 
compressed internally surprises me somewhat.

> But I'm also not sure you saw BOTH errors. 

yes

>My problem is with
> duplicity hanging after different kinds of errors on two
> machines.

both die in gzip.py
 
> So do BOTH of the following errors indicate the same thing
> of corrupted backup file(s)?

that's my guess. how can i be sure?

> 
>>> IOError: CRC check failed 0x43872847 != 0x7d505e15L
> 
>>> error: Error -3 while decompressing: invalid stored block lengths
> 
> (see my first mail for full error traces)
> 
>> PS: on errors please always send the complete
>> output of a maximum verbosity (-v9) duplicity
>> run that reproduces the error. obfuscate text
>> that you deem private in that log dump and
>> attach-post it to this mailing list.
> 
> sorry, i wasn't sure if it was useful to post
> so much data to the mailing list

ok, you missed your chance. again. can't do anything further without the log 
dump.

>> you may attach a compressed file if the mailing
>> list file size limit is reached.
> 
> i was looking at the next level below 9, but now
> I see maybe another clue on level 9.  problem is
> the verbose (-v9) log for first machine is 200MB.
> log for 2nd machine is 20MB.  even if i can post
> this somewhere, it's too much private info to
> clean up, I'm sorry. So if you know what parts you
> want, I am pleased to post them.

sorry, i can't. either you provide it, or not. you are free to obfuscate your 
local file name, as far as i am concerned. but i need to see the rest 
(duplicity command, volume names ...)

> For now, I post the parts right at the error below
> in this email also on bug tracker
> 
>> it shouldn't hang though. please file a bug
>> https://bugs.launchpad.net/duplicity
> 
> ok, i filed a bug.
> https://bugs.launchpad.net/duplicity/+bug/1405502
> 
> i can't hold on to my corrupted backups for much
> longer (maybe a week) due to space limitations, so
> please let me know what I can provide to help fix
> this problem at your earliest convenience

if you're interested to solve it, download the backup chains in question to a 
local drive. duplicity backup files have a proper naming with type/date. 
download everything from the last
  duplicity-full-signatures.*.gpg
to the
  latest file (duplicity-inc.*.gpg or duplicity-new-signatures.*.sigtar.gpg)
when the directory is listed via name/time ascending.

NOTE: files with an identical timestamp indicate one backup and hence belong 
together.
 
>> consider using "par2+", see manpage
>> http://duplicity.nongnu.org/duplicity.1.html
> 
> Thank you so much for your suggestion. I think i am interested
> in using par2 but the manpage doesn't tell much what it is.
> I read wikipedia and google a bit about it, seems a parity
> tool to help reconstruct corrupt archive files but still limited
> info about it, especially in context uf duplicity. I'd be glad
> to read more about it anyother places.

research what par2 does and then imagine duplicity is running it for you for 
each file up/downloaded transparently. it'll only complain if it cant restore 
something.
 
> But seems easy because only change is add "par2+" to begining
> of my backend config on duplicty cli.  Question is can I add
> par2+ to existing archives? Or must they be new?

you'll need to start with a new backup location as the old is missing the par2 
redundancy files.

> 
> Here is -v9 log for the error messages on TWO machines,
> different kinds of errors (but both hang).
> 
SNIP

need the full ones.. sorry.. ede