Re: [rdiff-backup-users] atomic increment files?

[Matthew Flaschen]

Marcel (Felix) Giannelia wrote:
> If a typical restore is only restoring a small part of the filesystem
>  and only going back a few days, you're right. But I wasn't even 
> concerned with restore operations -- I want the increment storage to
> be more efficient so that I can archive it quickly and easily.

As I see it, rdiff-backup should optimize for:

1. Restore speed.
2. Backup size
3. Backup speed

in that order.  It's not optimized for du performance.

> It doesn't strike me as non-obvious; reading an archive header on a
> file format that stores one (e.g. rar, zip, 7z as opposed to tar,
> which doesn't)

All three of these are proprietary, not that that's really the main point...

> has always seemed to go faster than enumerating the same list of
> files in the filesystem, when there are many files involved.

Enumeration speed is not the point either.  The point is how fast you
can restore and how fast you can backup.

> But when you're storing something that you *know* is going to be
> read-only and will never need to be modified again, then it makes
> more sense to store a nice index at the front (which is why CD's do
> that). Aside from unrecommended fiddling, people generally don't
> modify rdiff-backup's increment files so I think that would be a good
> application of a nice indexed archive.

They don't modify increment files /now/, because once you make an
increment it's done.  If rdiff-backup did things your way (one huge
file), then it would have to constantly modify the file.

> Creating the index would incur
> almost no extra time

How about modifying the index list? As Andrew noted, there is no
primitive for inserting or removing a line of a file (anywhere except
the end).

> increments could be stored without the extra
> slack space separate files cause, and typical restore operations
> wouldn't slow down by much (full restores from a long time ago would
> probably take longer, but they already take a while).

So everything slows down, but the most slowdown is on the part that's
already slowest?

Matt Flaschen