Re: [Lzip-bug] lzip and pipes

[Antonio Diaz Diaz]

Hi Daniel,

Daniel Klein wrote:
> lzip seems to somehow "block" I/O in a process feeding it data via pipe.

All processes block I/O feed to them via pipe if the producer is faster than 
the consumer. It is usually not noticed because most programs read data in 
small blocks. When compressing, lzip reads data in large blocks (one 
dictionary size at a time). It does this for efficiency reasons; each time 
it reads a block, it needs to make room for it with memmove. Reading the 
data in smaller blocks would probably make it slower, not faster.


> The "blocking intervals" get longer with higher compression levels.

This is because dictionary size grows (and compression speed decreases) with 
compression level.

I'll document this in the manual. Thanks.


Best regards,
Antonio.