[Duplicity-talk] Re: Re: Can one safely run multiple instances tothesame target?

[Eric B.]

> > There would definitely need to be a way to recover from an orphaned
> > lockfile.  Additionally, if you are considering adding semaphore ability
> > within duplicity itself, it might be an interesting thought to consider
> > adding in a --verify-after-backup option to verify within the same 
> > duplicity
> > execution.  ie: you would not want another instance of duplicity to be 
> > able
> > to grab the lock prior to being able to do a verify after the backup is
> > complete.
>
> --verify-after-backup is a good idea IMO.
>
> As for recovery - while it would be nice, the problem with advisory
> locking is that the policy with regards to any kind of
> recovery/fallback is going to be very dependent on situation.
>
> In the case of a portable mkdir lock, what would be your suggestion?

I'm not entirely sure.  One thing that one could conceivably do is something 
like "heartbeat" file whose timestamp is updated every XXX seconds.  If you 
set duplicity to produce a 5s heartbeat, a second instance would expect to 
see that file's timestamp be modified within 5 seconds.  If not, then the 
original duplicity likely failed.  Obviously, this is a very basic concept 
that I'm describing; I'd probably have the second instance wait a factor of 
the heartbeat timeout before it were to determine that the lock was stale. 
Of course, this would imply 2 simultaneous connections from duplicity to the 
backend; one to upload the file, and one to keep the heartbeat updated.  I 
don't know how easy it is to develop multi-threaded things in python ... i'm 
just brainstorming a little.

Otherwise, like you said, just have a simple option to retry for XXX 
timeout, and upon failure of grabbing the lock then to fail altogether.  In 
which case, it would take manual intervention to either delete the lock 
manually, or launch duplicity with the clean argument, or if you want to be 
even safer, clean --remove-orphaned-lockfile, etc...

Thanks,

Eric