[Duplicity-talk] some usage questions regarding full/incremental backups

[Mathijs Kwik]

Hi,

I currently use rdiff-backup to backup a few servers.
They all backup to 1 central backup-machine.
That machine thus has backup for all servers, including history.
This is all unencrypted, but this isn't an issue since the machine is
in the office and is secured by itself (disks are encrypted).
After all machines have backed-up to this central location, it uses
regular rsync to upload the archive to an off-site machine.
(I don't use rdiff-backup for this, since I don't need history of history :)
The off-site machine is in a fully secured, trusted environment.

Now,
I would like to change the off-site machine to a regular cheap
backup-hosting service (current location is _very_ expensive).
So I need encryption since I can't trust the backup-hosting provider.

So I found duplicity, it looks perfect, just the way I need it.

One thing that isn't clear to me is the full/incremental stuff.
Currently, using rdiff-backup, I can remove all stuff that's older than a month.
I saw duplicity can do this too, but the man-page mentions it won't
delete old stuff that has newer (incremental) stuff depending on it.
Maybe rdiff-backup has the same issue, didn't look into that yet.
So, if I only use incremental backups, the archive will fill-up more
and more even when cleaning old stuff.

So I need to use full backups every now and then.
Will this mean everything gets transfered fully?
Or is there some intelligent algorithm that justs 'merges' the
increments into a full backup?
Since our total backup is close to 1TB, I don't feel like transferring
everything every month or so.
Also, storage space might be an issue.
Say I want to keep a month of backups,
I would do a full backup every 2 weeks and clean up all but the last 2
full's right after that.
Does this mean that (right after doing the full backup, before
cleaning up), there are 3 full copies?
needing 3TB+ storage feels a bit much when the 'real' data is 'only' 1TB.
Am I miscalculating here? Or is there some smart strategy I can try?
All servers combined is about 1TB. daily changes are between 500Mb and 1Gb.

Like I said, maybe rdiff-backup has the same issue, but I can't
remember a full/incremental option on it.
As far as I remember rdiff-backup chose a reverse-incremental
strategy, so the most current backup is always full, and every
previous backup is decremented from that.

So I really hope some smart merging strategy can be used when doing full.
Otherwise a reverse/decremental strategy would be very helpful to save
space / bandwidth.

Any help is greatly appreciated.

Thanks,
Mathijs