Re: [Lzip-bug] [lzlib] Help with Scheme bindings (for GNU Guix)

[Antonio Diaz Diaz]

Hi Pierre,

Pierre Neidhardt wrote:
> My problem with LZ_decompress_finish is that the 2 functions above work
> on streams, so they are not supposed to finish anything, since they
> might just get called afterwards on the next chunk.

Then you must add an argument to the 2 functions to tell them that this 
is the last call and that they must:
  1) call LZ_(de)compress_finish
  2) continue reading until LZ_(de)compress_finished returns 1

http://www.nongnu.org/lzip/manual/lzlib_manual.html#Decompression-functions
Function: int LZ_decompress_finished ( struct LZ_Decoder * const decoder )
    Returns 1 if all the data have been read and 'LZ_decompress_close' 
can be safely called. Otherwise it returns 0.


> Note that the same decoder may be used on every call.

The same decoder MUST be used on every call if the compressed data spans 
more than one chunk. Else the decoder state between calls is lost.


> Question: is it OK to re-use a decoder?  Or is it recommended to
> re-create a decoder for every chunk (de)compression?

You must use the same decoder to decode all the chunks into which your 
system may split a lzip member[1]. You may use diferent decoders (or a 
reset or recreated decoder) for different lzip members.

[1] http://www.nongnu.org/lzip/manual/lzlib_manual.html#Data-format


> > If you don't call it, a truncated member may not be detected.
>
> Can you explain why?  If I read/write all the written data, is it
> possible that something would be missing?

If you write incomplete data (maybe because the file is truncated), and 
then call LZ_decompress_read until it returns 0 (without calling 
LZ_(de)compress_finish nor reading until LZ_(de)compress_finished 
returns 1), then the decompressed data produced will be also truncated. 
No error is reported because the decoder is waiting more input.


> > If you want it simple, you may even alternate write/read calls in a loop
> > without checking LZ_(de)compress_write_size. See this patch to bbexample.c:
>
> Thanks for this suggestion.  Indeed, I was wondering what's the need for
> checking the LZ_(de)compress_write_size.  Don't we have enough
> information by checking for the number of written bytes or for errors?

LZ_(de)compress_write_size is checked just for efficiency. To avoid 
writing when the input buffer is already full, or calling 
LZ_(de)compress_finish more than once.

Regards,
Antonio.