Re: tarlz --diff broken

[Antonio Diaz Diaz]

Devon Sean McCullough wrote:
> > The reason why I implemented --diff as I did is that the documentation
> > for GNU tar seems to imply that --directory is order sensitive only
> > for --create:
>
> LOL, now to test GNU tar for the same confusion, and if present, report
> that bug as well - as if I didn't have enough to do already!

As soon as I have the feature implemented in tarlz, I plan to report that 
the documentation for -C in GNU tar is incomplete. (Unless somebody else 
reports it first).

It seems that making -C order sensitive for anything except create and 
append is not so easy to do efficiently. The code for this in GNU tar is so 
intricate that I have needed to run it to check for sure for what operations 
-C is order sensitive. It will take me some time to figure out how to make 
it efficient without the complications of GNU tar.

> P.S. I use # (time tarlz ...) \
> | (pv --average-rate --bytes --eta --size $SIZE) \
> | (cd /exfat-sd-card && split ...)
> to make backups on pricey but handy SD Cards preformatted with EXFAT,
> universally readable but lacking large file support, after # (time tarlz
> --uncompressed ...) | wc -c for a quick $SIZE guesstimate. Any thoughts on
> --multi-volume or split support?

Lacking large file support is a problem nowadays because files larger than 2 
GiB are not uncommon. I plan to implement some sort of multivolume support 
in tarlz, but the way tarlz could split a tar file (putting at least one tar 
member in each lzip member) is not suitable for such file systems if large 
files are to be archived.

Have you tried to create the multivolume archive using lzip? It has the 
inconvenient that it does not keep the corresponcence between tar members 
and lzip members. But it has the advantage that it produces valid complete 
lzip files of a guaranteed maximum size. The command could be something like 
this:

(time tarlz --uncompressed ...) \
| (pv --average-rate --bytes --eta --size $SIZE) \
| (lzip -S 2GB -o /exfat-sd-card/volume_name -)

http://www.nongnu.org/lzip/manual/lzip_manual.html#Invoking-lzip
-S bytes
--volume-size=bytes
    When compressing, and '-c' has not been also specified, split the 
compressed output into several volume files with names 
'original_name00001.lz', 'original_name00002.lz', etc, and set the volume 
size limit to bytes. Input files are kept unchanged. Each volume is a 
complete, maybe multimember, lzip file. A small volume size may degrade 
compression ratio, so use it only when needed. Valid values range from 100 
kB to 4 EiB.

See also examples 10, 11, and 12 about how to create and restore multivolume 
compressed tar archives at 
http://www.nongnu.org/lzip/manual/lzip_manual.html#Examples

Best regards,
Antonio.