Re: [Lzip-bug] lzip memory & performance issue

[markk]

Hi,

Antonio Diaz Diaz wrote:
> As you can see here[1], dictionary sizes that are a power of two are
coded exactly. So lzip uses exactly 256MiB when you specify 256MiB. For
other values, lzip divides the space between any two powers of two in 16
intervals of equal size (wedges). For example, any value between 249MiB
and 255MiB will be rounded upwards to 256MiB.
> [1] http://www.nongnu.org/lzip/manual/lzip_manual.html#File-Format

Ah, thanks.

Regarding the performance difference between lzip and xz (at least on my
system), reducing --match-length did speed up lzip somewhat. With
--match-length=32, lzip took about 13:50 to compress the 633169920-byte
file mentioned in my previous message.

That's down from about 19 minutes with the (default?) --match-length=273,
but it's still about twice as long as xz. (Compression ratio reduces when
the match length is lower.)

For that specific file, lzip -9 compressed size was 334811557 bytes (match
length 273). Match length 200 reduced the size to 334811557.

With match length 32:  335674532
With match length 64:  335105285
With match length 128: 334810420
With match length 200: 334760646

So 273 isn't necessarily optimal for all files.

Interestingly, xz -9e uses a default match length (it calls the option
"nice") of 64. But for some reason the xz-compressed file was still
smaller than the lzip one.