Re: [Duplicity-talk] Orphaned signature files, backup broken?

[Marc Hoersken]

On 12.06.2014 13:57, edgar.soldin wrote:
> you know that you can verify past backups via -t (time) option? duply 
> essentially routes that through
> see duplicity manpage
>  http://duplicity.nongnu.org/duplicity.1.html

No, I didn't know. Running verify with -t and a timestamp of an orphaned
signature file results in the following output:

sudo duply picard-homes verify -t "20140601T042703Z"
Start duply v1.5.5.5, time is 2014-06-12 14:06:38.
Using profile '/etc/duply/picard-homes'.
Using installed duplicity version 0.6.24, python 2.7.3, gpg 1.4.12
(Home: ~/.gnupg), awk 'GNU Awk 4.0.1', bash '4.2.37(1)-release
(x86_64-pc-linux-gnu)'.
Using configured key '...' as signing key.
Test - Encrypt to ... & Sign with ... (OK)
Test - Decrypt (OK)
Test - Compare (OK)
Cleanup - Delete '/tmp/duply.11885.1402574799_*'(OK)

--- Start running command VERIFY at 14:06:39.253 ---
LFTP version is 4.3.6
Reading globbing filelist /etc/duply/picard-homes/exclude
Local and Remote metadata are synchronized, no sync needed.
Warning, found the following local orphaned signature files:
duplicity-new-signatures.20140513T043031Z.to.20140524T095144Z.sigtar.gz
duplicity-new-signatures.20140513T043031Z.to.20140524T095127Z.sigtar.gz
duplicity-new-signatures.20140524T095144Z.to.20140525T043012Z.sigtar.gz
duplicity-new-signatures.20140525T043012Z.to.20140526T042651Z.sigtar.gz
duplicity-new-signatures.20140526T042651Z.to.20140527T042712Z.sigtar.gz
duplicity-new-signatures.20140527T042712Z.to.20140528T042707Z.sigtar.gz
duplicity-new-signatures.20140528T042707Z.to.20140529T042657Z.sigtar.gz
duplicity-new-signatures.20140529T042657Z.to.20140530T042709Z.sigtar.gz
duplicity-new-signatures.20140530T042709Z.to.20140531T042656Z.sigtar.gz
duplicity-new-signatures.20140531T042656Z.to.20140601T042703Z.sigtar.gz
duplicity-new-signatures.20140601T042703Z.to.20140602T042705Z.sigtar.gz
duplicity-new-signatures.20140602T042705Z.to.20140603T042700Z.sigtar.gz
duplicity-new-signatures.20140603T042700Z.to.20140604T042700Z.sigtar.gz
duplicity-new-signatures.20140604T042700Z.to.20140605T042706Z.sigtar.gz
duplicity-new-signatures.20140605T042706Z.to.20140606T042703Z.sigtar.gz
duplicity-new-signatures.20140606T042703Z.to.20140607T042711Z.sigtar.gz
duplicity-new-signatures.20140607T042711Z.to.20140608T042816Z.sigtar.gz
duplicity-new-signatures.20140608T042816Z.to.20140609T042703Z.sigtar.gz
duplicity-new-signatures.20140609T042703Z.to.20140610T042656Z.sigtar.gz
duplicity-new-signatures.20140610T042656Z.to.20140611T042745Z.sigtar.gz
Warning, found the following remote orphaned signature files:
duplicity-new-signatures.20140513T043031Z.to.20140524T095143Z.sigtar.gpg
duplicity-new-signatures.20140513T043031Z.to.20140524T095144Z.sigtar.gpg
duplicity-new-signatures.20140524T095144Z.to.20140525T043012Z.sigtar.gpg
duplicity-new-signatures.20140525T043012Z.to.20140526T042651Z.sigtar.gpg
duplicity-new-signatures.20140526T042651Z.to.20140527T042712Z.sigtar.gpg
duplicity-new-signatures.20140527T042712Z.to.20140528T042707Z.sigtar.gpg
duplicity-new-signatures.20140528T042707Z.to.20140529T042657Z.sigtar.gpg
duplicity-new-signatures.20140529T042657Z.to.20140530T042709Z.sigtar.gpg
duplicity-new-signatures.20140530T042709Z.to.20140531T042656Z.sigtar.gpg
duplicity-new-signatures.20140531T042656Z.to.20140601T042703Z.sigtar.gpg
duplicity-new-signatures.20140601T042703Z.to.20140602T042705Z.sigtar.gpg
duplicity-new-signatures.20140602T042705Z.to.20140603T042700Z.sigtar.gpg
duplicity-new-signatures.20140603T042700Z.to.20140604T042700Z.sigtar.gpg
duplicity-new-signatures.20140604T042700Z.to.20140605T042706Z.sigtar.gpg
duplicity-new-signatures.20140605T042706Z.to.20140606T042703Z.sigtar.gpg
duplicity-new-signatures.20140606T042703Z.to.20140607T042711Z.sigtar.gpg
duplicity-new-signatures.20140607T042711Z.to.20140608T042816Z.sigtar.gpg
duplicity-new-signatures.20140608T042816Z.to.20140609T042703Z.sigtar.gpg
duplicity-new-signatures.20140609T042703Z.to.20140610T042656Z.sigtar.gpg
duplicity-new-signatures.20140610T042656Z.to.20140611T042745Z.sigtar.gpg
Warning, found incomplete backup sets, probably left from aborted session
Last full backup date: Wed Jun 11 22:05:19 2014
Invalid data - SHA1 hash mismatch for file:
 duplicity-inc.20140513T043031Z.to.20140524T095144Z.vol1.difftar.gpg
 Calculated hash: df106f05caf3e2a9fc4b186dc9c13cb221264dee
 Manifest hash: 4971120b3d753498cb8a9f8ed9b685abeb9baef4

And it gets stuck there with many gpg processes, see attached ps output.

> duplicity/duply has a cleanup command in case that files get leftover or from 
> incomplete backups (interrupted & unresumed). we also had/have a bug that 
> leaves old signatures lying around.

I am currently not using the cleanup or purge commands since I am not
sure if those files are actually needed, because they belong to the
backup archives since the 13.05.2014.

proc.log
Description: Text document