[rdiff-backup-users] Resuming failed backups

[Josh Nisly]

I have a number of clients backing up over slow internet links. Because 
of the inherent flakiness of public internet connections, it can be 
frustrating when the client dumps a large set of new data in a folder 
that is being backed up. If the backup fails partway through, 
rdiff-backup regresses the destination to the last good backup, throwing 
away any data the client has uploaded. Since rdiff-backup now supports 
resuming initial backups, this seems like the Achilles' heel of 
rdiff-backup over slow connections.

I believe strongly in rdiff-backup's idea of consistent "snapshots"; I 
don't think that it should tie off a partial backup. Instead, I think it 
should simply keep the uploaded files outside of the backup set and use 
them in the next backup.

Here is what I propose: when regressing a repository prior to a backup, 
rdiff-backup takes all "new" files (files that have been added during 
the failed backup) and moves them to a temporary location inside of the 
rdiff-backup-data folder. Then, when the backup runs, if it encounters a 
new file, it first checks to see if the file exists in this temporary 
location, and if it does, it diffs against that file (or moves it to the 
target location, then diffs; I don't know which would be easier). At the 
end of any backup run, rdiff-backup empties out this folder.

Thoughts/reactions?

Thanks,
JoshN