[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Duplicity-talk] Weird error message after incremental backup of large d
From: |
Jakob Bohm |
Subject: |
[Duplicity-talk] Weird error message after incremental backup of large drive with a bunch of changed files |
Date: |
Tue, 14 Mar 2023 21:54:12 +0100 |
User-agent: |
Mozilla/5.0 (Windows NT 6.3; Win64; x64; rv:5.2) Goanna/20221030 Epyrus/2.0.0 |
Dear group,
I have set up some scripts to do various parts of a full system backup
via duplicity to a geographically close S3 bucket (AWS Stockholm),
however for the largest drive, I occasionally experience hangs/errors
near the end of each backup, with the progress display wobbling between
"stalled" and less than 30 minutes left (this time I observed as low as
16 minutes at one point).
Then a day into the stall/short time phase, I received to following
error message (redacted bucket name, mountpoints etc.):
Attempt of put Nr. 1 failed. S3UploadFailedError: Failed to upload
/duplicitypart/tmp/commonprefix-partname/duplicity-g3cvly63-tempdir/duplicity_temp.1/commonprefix-partname_duplicity-inc.20230212T043018Z.to.20230311T203857Z.vol1501.difftar.gpg.vol000+200.par2
to
bucketname/partname/commonprefix-partname_duplicity-inc.20230212T043018Z.to.20230311T203857Z.vol1501.difftar.gpg.vol000+200.par2:
An error occurred (RequestTimeTooSkewed) when calling the
CreateMultipartUpload operation: The difference between the request time
and the current time is too large.
After the message there were some small increases in the GB counter in
the progress bar
ls report after killing duplicity:
-rw-r--r-- 1 root root 21336776 Mar 13 18:33
/duplicitypart/tmp/commonprefix-partname/duplicity-g3cvly63-tempdir/duplicity_temp.1/commonprefix-partname_duplicity-inc.20230212T043018Z.to.20230311T203857Z.vol1501.difftar.gpg.vol000+200.par2
Currently invoking duplicity 1.2.1 (patched) with command line
duplicity incremental --name commonprefix-partname \
--archive-dir /duplicitypart/archive/commonprefix-partname \
--asynchronous-upload \
--file-prefix commonprefix-partname_ \
--tempdir /duplicitypart/tmp/commonprefix-partname \
--verbosity notice \
--progress \
--log-file
/duplicitypart/tmp/commonprefix-partname/log_20230311T20_38_52.log \
--gpg-options '--homedir /configdir/.gnupg --compress-algo=bzip2' \
--encrypt-secret-keyring /configdir/.gnupg/secret.gpg \
--encrypt-key 1234567890ABCDEF1234567890ABCDEF12345678 \
--sign-key 1234567890ABCDEF1234567890ABCDEF12345678 \
--hidden-encrypt-key 1234567890ABCDEF1234567890ABCDEF12345678 \
--exclude-other-filesystems \
--full-if-older-than 3M \
--s3-use-multiprocessing \
--numeric-owner \
/partname \
par2+boto3+s3://bucketname/partname
OS: Debian GNU/Linux 11.7 (bullseye) with Python 3.9.2, python-boto3
version 1.13.14-1
I suspect a relationship with issue #254, and as you see, I have
incorporated some of the workarounds into the command line.
What are the appropriate troubleshooting steps?
Enjoy
Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded
- [Duplicity-talk] Weird error message after incremental backup of large drive with a bunch of changed files,
Jakob Bohm <=