[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Duplicity-talk] failed to upload before termination, restarting, re
From: |
edgar . soldin |
Subject: |
Re: [Duplicity-talk] failed to upload before termination, restarting, restore SHA1 hash mismatch |
Date: |
Fri, 17 Apr 2015 12:50:24 +0200 |
User-agent: |
Mozilla/5.0 (Windows NT 5.1; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 |
On 17.04.2015 12:46, Maximilian Bloch wrote:
> Hi there,
>
> I'm new here, nice to meet everyone. First off, I am pretty happy with
> duplicity, which has been working for me for the past two years.
>
> I have a current issue though and no solution. I searched through the
> mailing-list archive and couldn't find anything to help me. Sorry if
> this has been asked before.
>
> I am doing automated daily backups:
>
> duplicity \
> --full-if-older-than 14D \
> --encrypt-key ${GPG_KEY} \
> --sign-key ${GPG_KEY} \
> --exclude-regexp '\/te?mp\/' \
> --exclude-regexp '\/cache\/' \
> --include '/var/www' \
> --exclude '**' \
> / s3+http://xxxx-var-www
>
>
> My last full backup(s) seem to have failed (Backup taking longer than
> 24h and conflicting with the next backup script?), then resumed the next
> day with further errors, resumed/restarted on the third day with no
> errors. Restoring the backup is failing due to a corrupted volume (SHA1
> hash mismatch for file ..vol907.difftar.gpg). I have noticed similar
> issues with my last two full backups. My backup consists of 1043 volumes
> à 25MB.
>
> Does anybody have a solution to get the next backups to work properly?
> bigger volume sizes? More time until script is run again?
>
> Here are some details of duplicity output:
>
> Full backup take 1:
> Sun Apr 12, Sorry, no output, I must have trashed it because it
> contained no errors.
>
> Full backup take 2:
> Local and Remote metadata are synchronized, no sync needed.
> Last full backup date: Sun Apr 12 07:12:11 2015
> No old backup sets found, nothing deleted.
> Local and Remote metadata are synchronized, no sync needed.
> Last full backup left a partial set, restarting.
> Last full backup date: Sun Apr 12 07:12:11 2015
> Reuse configured PASSPHRASE as SIGN_PASSPHRASE
> RESTART: Volumes 907 to 908 failed to upload before termination.
> Restarting backup at volume 907.
> Restarting after volume 906, file var/www/xxxx.png, block 13
> File duplicity-full.20150412T051211Z.vol914.difftar.gpg was corrupted
> during upload.
>
>
> Full backup take 3 (note it takes longer than 24h):
>
> Local and Remote metadata are synchronized, no sync needed.
> Last full backup date: Sat Mar 28 07:09:23 2015
> No old backup sets found, nothing deleted.
> Local and Remote metadata are synchronized, no sync needed.
> Last full backup date: Sat Mar 28 07:09:23 2015
> Last full backup is too old, forcing full backup
> Reuse configured PASSPHRASE as SIGN_PASSPHRASE
> --------------[ Backup Statistics ]--------------
> StartTime 1428815542.46 (Sun Apr 12 07:12:22 2015)
> EndTime 1428911965.18 (Mon Apr 13 09:59:25 2015)
> ElapsedTime 96422.72 (26 hours 47 minutes 2.72 seconds)
> SourceFiles 150833
> SourceFileSize 28580154097 (26.6 GB)
> NewFiles 150833
> NewFileSize 28580154097 (26.6 GB)
> DeletedFiles 0
> ChangedFiles 0
> ChangedFileSize 0 (0 bytes)
> ChangedDeltaSize 0 (0 bytes)
> DeltaEntries 150833
> RawDeltaSize 28490687563 (26.5 GB)
> TotalDestinationSizeChange 27385520785 (25.5 GB)
> Errors 0
>
>
> Restoring attempt:
> [...]
> Processed volume 908 of 1048
> Invalid data - SHA1 hash mismatch for file:
> duplicity-full.20150412T051211Z.vol907.difftar.gpg
> Calculated hash: 3b332923613018d50d4b73d0b2831c72c36f4d74
> Manifest hash: f9f8fa5e8bd5a07bb17228c83e13cf9f6f2adcdb
>
> ^ the restore script hangs here
>
>
> Here is some output of my previous to last full backup attempt, which
> also failed:
>
> Local and Remote metadata are synchronized, no sync needed.
> Last full backup date: Sat Mar 14 06:29:20 2015
> No old backup sets found, nothing deleted.
> Local and Remote metadata are synchronized, no sync needed.
> Last full backup date: Sat Mar 14 06:29:20 2015
> Last full backup is too old, forcing full backup
> Reuse configured PASSPHRASE as SIGN_PASSPHRASE
> Upload
> 's3+http://xxxx-var-www/duplicity-full-signatures.20150328T060923Z.sigtar.gpg'
> failed (attempt #1, reason: S3ResponseError: S3ResponseError: 400 Bad
> Request
> <?xml version="1.0" encoding="UTF-8"?>
> <Error><Code>BadDigest</Code><Message>The Content-MD5 you specified did
> not match what we
> received.</Message><ExpectedDigest>oKyY+wDSBIKvBXzNFupGIQ==</ExpectedDigest><CalculatedDigest>zHtMZsbKINhRo9flUUjXPA==</CalculatedDigest><RequestId>4177190AB50613BA</RequestId><HostId>+hN7JpYX6auLQ63LegVOHtpxBk9ZHFfYU/92HucGXhFuIRKyScdWruXujslIFIb5dh1tfoeFcp8=</HostId></Error>)
> Traceback (most recent call last):
> File "/usr/bin/duplicity", line 1404, in <module>
> with_tempdir(main)
> File "/usr/bin/duplicity", line 1397, in with_tempdir
> fn()
> File "/usr/bin/duplicity", line 1367, in main
> full_backup(col_stats)
> File "/usr/bin/duplicity", line 506, in full_backup
> sig_outfp.to_remote()
> File "/usr/lib/python2.7/dist-packages/duplicity/dup_temp.py", line
> 184, in to_remote
> globals.backend.move(tgt) address@hidden
> File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line
> 364, in move
> source_path.delete()
> File "/usr/lib/python2.7/dist-packages/duplicity/path.py", line 567,
> in delete
> util.ignore_missing(os.unlink, self.name)
> File "/usr/lib/python2.7/dist-packages/duplicity/util.py", line 116,
> in ignore_missing
> fn(filename)
> OSError: [Errno 2] No such file or directory:
> '/root/.cache/duplicity/16e0b913754120e86533128ea399d494/duplicity-full-signatures.20150328T060923Z.sigtar.gpg'
>
>
what's your duplicity version? ..ede/duply.net
- [Duplicity-talk] failed to upload before termination, restarting, restore SHA1 hash mismatch, Maximilian Bloch, 2015/04/17
- Re: [Duplicity-talk] failed to upload before termination, restarting, restore SHA1 hash mismatch,
edgar . soldin <=
- Re: [Duplicity-talk] failed to upload before termination, restarting, restore SHA1 hash mismatch, Maximilian Bloch, 2015/04/17
- Re: [Duplicity-talk] failed to upload before termination, restarting, restore SHA1 hash mismatch, edgar . soldin, 2015/04/17
- Re: [Duplicity-talk] failed to upload before termination, restarting, restore SHA1 hash mismatch, Maximilian Bloch, 2015/04/17
- Re: [Duplicity-talk] failed to upload before termination, restarting, restore SHA1 hash mismatch, Maximilian Bloch, 2015/04/27
- Re: [Duplicity-talk] failed to upload before termination, restarting, restore SHA1 hash mismatch, Kenneth Loafman, 2015/04/27
- Re: [Duplicity-talk] failed to upload before termination, restarting, restore SHA1 hash mismatch, edgar . soldin, 2015/04/27