Re: [Duplicity-talk] empty cache folder

[edgar . soldin]

@Ken, Mike: there are points below on which i wasn't sure. you could shed some 
insight on it?

On 26.12.2012 15:33, Druzee wrote:
> I did, Yes, I understand that the signature and manifest files are a must, 

not generally. 

the signature is pretty much optional. it's needed only for file listing. it 
contains information also avail in the volumes but provides it without the need 
to download them (bandwidth efficiency).

essential is the manifest. the name is probably chosen to reflect
http://en.wikipedia.org/wiki/Manifest_%28transportation%29
it contains administrative information on where what is stored (start,end block 
in a volume, hash).

>but what's the logic on where to store them? locally in .cache, on the remote 
>folder, both? 

both, 
1. unencrypted locally in archive folder under .cache
2. encrypted on remote

i am not very familiar with duplicity creating backup data internals, but i 
imagine that the data is stored locally in archive dir and later 
encrypted/gzipped/transported to remote.

i guess on a restart the local archive dir manifest is used to determine where 
to continue. right Ken?

>Why are they sometimes only in one place and not the other ?

probably the backup was interrupted before the file reached the second 
location. 

> 
> Does missing manifest and/or signature files render the backup worthless? 

manifest, yes.

>I know i can restore the files manually, but will the splitted files between 
>volumes restore ok ?

there is an howto by Michael Terry
https://live.gnome.org/DejaDup/Help/Restore/WorstCase#Restoring_by_Hand

it reads like it should be perfectly possible. Michael?

> 
> the command lines are written in the logs, it's as simple as
> 
> duplicity full . s3+http://bucket/folder -vi --name Pictures 
> --s3-unencrypted-connection

yeah, saw them.. just being thorough ;)

> BTW, i tried undeleting files in the .cache/duplicity/Pictures folder. no 
> luck there.

my current guess is that you have some weird routine that cleans out ~/.cache 
for you from time to time. possible?
you could move archive dir to some other folder, just to make sure that's not 
it. see manpage. 


so far.. ede/duply.net

> 
> 
> 
> On Wed, Dec 26, 2012 at 4:21 PM, <address@hidden <mailto:address@hidden>> 
> wrote:
> 
>     ok, regarding that - did you see the answer from the author and my 
> questions regarding it?
>     http://article.gmane.org/gmane.comp.sysutils.backup.duplicity.general/5763
>     that should shed light on the issue.
> 
>     not having manifest/signature without deleting them is another thing 
> though. you should probably start with a new full and keep an eye out when 
> they disappear. keep the logs, maybe ran in debug verbosity, for the time 
> being.
> 
>     are you doing manual backups or using cron? what are your command lines 
> like?
> 
>     ..ede/duply.net <http://duply.net>
> 
>     On 26.12.2012 15:11, Druzee wrote:
>     > No.
>     >
>     > I tried stopping and resuming the backup on that machine and it works. 
> which leaves me even more clueless on why the backup does not resume on the 
> other machine.
>     >
>     >
>     >
>     > On Wed, Dec 26, 2012 at 4:03 PM, <address@hidden 
> <mailto:address@hidden> <mailto:address@hidden <mailto:address@hidden>>> 
> wrote:
>     >
>     >     that was on the 500+ backup, what about the 1000+ one? ever did a 
> cleanup or remove-* on that?
>     >
>     >     ..ede
>     >
>     >     On 25.12.2012 15 <tel:25.12.2012%2015> <tel:25.12.2012%2015>:50, 
> Druzee wrote:
>     >     > Nope... no cleaning.
>     >     >
>     >     > Like i said, duplicity stopped responding (after resuming from a 
> notebook suspend, moved to a different network), i killed it, when i reran 
> the command line i suddently saw that it starts running from volume 1 again...
>     >     >
>     >     >
>     >     >
>     >     > On Tue, Dec 25, 2012 at 4:44 PM, <address@hidden 
> <mailto:address@hidden> <mailto:address@hidden <mailto:address@hidden>> 
> <mailto:address@hidden <mailto:address@hidden> <mailto:address@hidden 
> <mailto:address@hidden>>>> wrote:
>     >     >
>     >     >     On 25.12.2012 15 <tel:25.12.2012%2015> <tel:25.12.2012%2015> 
> <tel:25.12.2012%2015>:34, Druzee wrote:
>     >     >     >>does that backup (1000+) verify? ..ede
>     >     >     >>
>     >     >     >>On 25.12.2012 14 <tel:25.12.2012%2014> 
> <tel:25.12.2012%2014> <tel:25.12.2012%2014>:50, Druzee wrote:
>     >     >     >>>/ Ineed the folder on S3 does not have a "manifest" (or 
> "sigtar") file like i /
>     >     >     >>>/ have in a different one. I am 100% sure that i did not 
> erase any file from /
>     >     >     >>>/ S3. What even more worries me is that on a different, 
> currently active backup /
>     >     >     >>>/ set that i have with over 1000 volumes the manifest file 
> is also missing./
>     >     >     >
>     >     >     > I am no where near that machine and the backup is still 
> active. but i did try listing the files and it doesn't look good.
>     >     >     >
>     >     >     > Using archive dir: 
> /home/xxxx/.cache/duplicity/146a0d0ed53e8c25f4437132c5de0087
>     >     >     > Using backup name: 146a0d0ed53e8c25f4437132c5de0087
>     >     >     > Import of duplicity.backends.localbackend Succeeded
>     >     >     > Import of duplicity.backends.webdavbackend Succeeded
>     >     >     > Import of duplicity.backends.ftpbackend Succeeded
>     >     >     > Import of duplicity.backends.sshbackend Succeeded
>     >     >     > Import of duplicity.backends.tahoebackend Succeeded
>     >     >     > Import of duplicity.backends.cloudfilesbackend Succeeded
>     >     >     > Import of duplicity.backends.ftpsbackend Succeeded
>     >     >     > Import of duplicity.backends.botobackend Succeeded
>     >     >     > Import of duplicity.backends.gdocsbackend Succeeded
>     >     >     > Import of duplicity.backends.hsibackend Succeeded
>     >     >     > Import of duplicity.backends.imapbackend Succeeded
>     >     >     > Import of duplicity.backends.u1backend Failed: No module 
> named httplib2
>     >     >     > Import of duplicity.backends.rsyncbackend Succeeded
>     >     >     > Main action: list-current
>     >     >     > 
> ================================================================================
>     >     >     > duplicity 0.6.20 (October 28, 2012)
>     >     >     > Args: /usr/bin/duplicity list-current-files 
> s3://bucket/Pictures/ -v9
>     >     >     > Linux null 3.6.10-1-ARCH #1 SMP PREEMPT Tue Dec 11 09:40:17 
> CET 2012 x86_64
>     >     >     > /usr/bin/python2 2.7.3 (default, Dec 12 2012, 07:12:24)
>     >     >     > [GCC 4.7.2]
>     >     >     > 
> ================================================================================
>     >     >     > Local and Remote metadata are synchronized, no sync needed.
>     >     >     > 0 files exist on backend
>     >     >     > 0 files exist in cache
>     >     >     > Extracting backup chains from list of files: []
>     >     >     > Last full backup date: none
>     >     >     > Collection Status
>     >     >     > -----------------
>     >     >     > Connecting with backend: BotoBackend
>     >     >     > Archive dir: 
> /home/xxxx/.cache/duplicity/146a0d0ed53e8c25f4437132c5de0087
>     >     >     >
>     >     >     > Found 0 secondary backup chains.
>     >     >     > No backup chains with active signatures found
>     >     >     > No orphaned or incomplete backup sets found.
>     >     >     > Using temporary directory /tmp/duplicity-obOjuZ-tempdir
>     >     >     > Traceback (most recent call last):
>     >     >     >   File "/usr/bin/duplicity", line 1403, in <module>
>     >     >     >     with_tempdir(main)
>     >     >     >   File "/usr/bin/duplicity", line 1396, in with_tempdir
>     >     >     >     fn()
>     >     >     >   File "/usr/bin/duplicity", line 1334, in main
>     >     >     >     list_current(col_stats)
>     >     >     >   File "/usr/bin/duplicity", line 599, in list_current
>     >     >     >     sig_chain = col_stats.get_signature_chain_at_time(time)
>     >     >     >   File 
> "/usr/lib/python2.7/site-packages/duplicity/collections.py", line 979, in 
> get_signature_chain_at_time
>     >     >     >     raise CollectionsError("No signature chains found")
>     >     >     > CollectionsError: No signature chains found
>     >     >     >
>     >     >
>     >     >     nothing to worry for now. the metadata is only there so 
> incrementing and listing files can be done without downloading the *big* 
> volumes again.
>     >     >     if the volumes are proper you should be able to restore 
> perfectly.
>     >     >
>     >     >     did you by any chance ran cleanup --extra-clean or something 
> like that?
>     >     >
>     >     >     ..ede
>     >     >
>     >     >     _______________________________________________
>     >     >     Duplicity-talk mailing list
>     >     >     address@hidden <mailto:address@hidden> <mailto:address@hidden 
> <mailto:address@hidden>> <mailto:address@hidden <mailto:address@hidden> 
> <mailto:address@hidden <mailto:address@hidden>>>
>     >     >     https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>     >     >
>     >     >
>     >     >
>     >     >
>     >     > _______________________________________________
>     >     > Duplicity-talk mailing list
>     >     > address@hidden <mailto:address@hidden> <mailto:address@hidden 
> <mailto:address@hidden>>
>     >     > https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>     >     >
>     >
>     >     _______________________________________________
>     >     Duplicity-talk mailing list
>     >     address@hidden <mailto:address@hidden> <mailto:address@hidden 
> <mailto:address@hidden>>
>     >     https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>     >
>     >
>     >
>     >
>     > _______________________________________________
>     > Duplicity-talk mailing list
>     > address@hidden <mailto:address@hidden>
>     > https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>     >
> 
>     _______________________________________________
>     Duplicity-talk mailing list
>     address@hidden <mailto:address@hidden>
>     https://lists.nongnu.org/mailman/listinfo/duplicity-talk
> 
> 
> 
> 
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>