Re: [Duplicity-talk] duply: Incrementals not shown

[Florian Lindner]

Hey,

Am 03.01.2017 um 12:08 schrieb edgar.soldin--- via Duplicity-talk:
> hi Florian, comment inline below
> 
> On 01.01.2017 15:30, Florian Lindner via Duplicity-talk wrote:
>> Hello,
>>
>> Everybody have a Happy New Year!
>>
>> I use duplicity though Duply on Debian Testing and have a weird problem. I 
>> have two profiles, that differ only by the SOURCE setting:
>>
>> address@hidden ~ # grep -v "^#" .duply/home/conf | grep -v "^$"
>>
>> GPG_KEY='disabled'
>> GPG_PW='_GPG_PASSWORD_'
>> TARGET='scp://address@hidden:2022/Backup/home'
>> SOURCE='/home'
>> MAX_AGE=1M
>> MAX_FULLBKP_AGE=2W
>> DUPL_PARAMS="$DUPL_PARAMS --full-if-older-than $MAX_FULLBKP_AGE "
>> VOLSIZE=25
>> DUPL_PARAMS="$DUPL_PARAMS --volsize $VOLSIZE "
>> VERBOSITY=5
>>
>> The other profile works perfectly. duply status shows all backups I did.
>>
>> However, at the home profile does not show any incremental backups I did 
>> after the initial full backup.
>>
>> I do a successfull incremental backup:
>>
> SNIP
>>
>>
>>
>> So everything looks ok, but:
>>
>> # duply home status
>> Start duply v1.11.3, time is 2017-01-01 15:27:29.
> SNIP
> 
>> Synchronizing remote metadata to local cache...
>> Deleting local 
>> /root/.cache/duplicity/duply_home/duplicity-inc.20161231T112717Z.to.20170101T140824Z.manifest
>>  (not authoritative at backend).
>> Deleting local 
>> /root/.cache/duplicity/duply_home/duplicity-new-signatures.20161231T112717Z.to.20170101T140824Z.sigtar.gz
>>  (not authoritative at backend).
> 
> see here your locally cached inremental get's deleted because duplicity does 
> not seem to see it on your backend.
> 
> can you browse there and check if the files really ended up there? try 
> switching to the pexpect+scp:// backend and see if the error resolves. 
> paramiko sometimes has issues listing files on the backend

Files on TARGET:

address@hidden /home/venus/Backup/home # ll *sig*
-rw------- 1 venus venus 661868201 31. Dez 17:21 
duplicity-full-signatures.20161231T112717Z.sigtar.gz
-rw------- 1 venus venus  11327438 31. Dez 17:36 
duplicity-new-signatures.20161231T112717Z.to.20161231T163047Z.sigtar.gz
-rw------- 1 venus venus  11345015 31. Dez 17:45 
duplicity-new-signatures.20161231T112717Z.to.20161231T163906Z.sigtar.gz
-rw------- 1 venus venus  11763366  1. Jan 02:11 
duplicity-new-signatures.20161231T112717Z.to.20170101T010504Z.sigtar.gz
-rw------- 1 venus venus  12878011  1. Jan 15:14 
duplicity-new-signatures.20161231T112717Z.to.20170101T140824Z.sigtar.gz
-rw------- 1 venus venus  13268770  1. Jan 20:20 
duplicity-new-signatures.20161231T112717Z.to.20170101T191037Z.sigtar.gz
-rw------- 1 venus venus  13271921  1. Jan 20:32 
duplicity-new-signatures.20161231T112717Z.to.20170101T192657Z.sigtar.gz
-rw------- 1 venus venus  13526360  2. Jan 02:11 
duplicity-new-signatures.20161231T112717Z.to.20170102T010504Z.sigtar.gz
-rw------- 1 venus venus  14738759  3. Jan 02:13 
duplicity-new-signatures.20161231T112717Z.to.20170103T010511Z.sigtar.gz
-rw------- 1 venus venus  15035322  3. Jan 13:26 
duplicity-new-signatures.20161231T112717Z.to.20170103T121953Z.sigtar.gz

The ones in the night are from the backup cronjob, the last one 13:26 is from 
the backup I just did. So everything looks in order (for me).

Trying pexpect+scp:


# duply home status
[...]
Synchronizing remote metadata to local cache...
Copying duplicity-inc.20161231T112717Z.to.20161231T163047Z.manifest to local 
cache.
[...]
Copying duplicity-new-signatures.20161231T112717Z.to.20161231T163047Z.sigtar.gz 
to local cache.
[...]
Copying duplicity-new-signatures.20161231T112717Z.to.20170103T010511Z.sigtar.gz 
to local cache.
Warning, found the following local orphaned signature files:
duplicity-new-signatures.20161231T112717Z.to.20170101T192657Z.sigtar.gz
duplicity-new-signatures.20161231T112717Z.to.20170101T010504Z.sigtar.gz
duplicity-new-signatures.20161231T112717Z.to.20170101T140824Z.sigtar.gz
duplicity-new-signatures.20161231T112717Z.to.20170101T191037Z.sigtar.gz
duplicity-new-signatures.20161231T112717Z.to.20170103T121953Z.sigtar.gz
duplicity-new-signatures.20161231T112717Z.to.20161231T163047Z.sigtar.gz
duplicity-new-signatures.20161231T112717Z.to.20161231T163906Z.sigtar.gz
duplicity-new-signatures.20161231T112717Z.to.20170102T010504Z.sigtar.gz
Warning, found the following remote orphaned signature files:
duplicity-new-signatures.20161231T112717Z.to.20161231T163906Z.sigtar.gz
duplicity-new-signatures.20161231T112717Z.to.20170101T010504Z.sigtar.gz
duplicity-new-signatures.20161231T112717Z.to.20170101T140824Z.sigtar.gz
duplicity-new-signatures.20161231T112717Z.to.20170101T191037Z.sigtar.gz
duplicity-new-signatures.20161231T112717Z.to.20170101T192657Z.sigtar.gz
duplicity-new-signatures.20161231T112717Z.to.20170102T010504Z.sigtar.gz
duplicity-new-signatures.20161231T112717Z.to.20170103T010511Z.sigtar.gz
duplicity-new-signatures.20161231T112717Z.to.20170103T121953Z.sigtar.gz
Warning, found signatures but no corresponding backup files
Last full backup date: Sat Dec 31 12:27:17 2016
Collection Status
-----------------
Connecting with backend: BackendWrapper
Archive dir: /root/.cache/duplicity/duply_home

Found 1 secondary backup chain.
Secondary chain 1 of 1:
-------------------------
Chain start time: Sat Dec 31 12:27:17 2016
Chain end time: Tue Jan  3 13:19:53 2017
Number of contained backup sets: 2
Total number of contained volumes: 1282
 Type of backup set:                            Time:      Num volumes:
                Full         Sat Dec 31 12:27:17 2016              1280
         Incremental         Tue Jan  3 13:19:53 2017                 2
-------------------------

Ok, I think this is ok, because the incremental backups didn't see the previous 
incremental backup. duply home cleanup --force fixed that, except
Warning, found signatures but no corresponding backup files. I will do a new 
complete backup later.

So I assume everything is working fine now.

Do you have any idea why only one profile is affected while the other one, same 
settings, same TARGET is not troublesome?

Anyway, big thanks to you!

Florian
Florian