Re: File ends unexpectedly at pos XXX when compressing

[Antonio Diaz Diaz]

Hello Scott,

Scott Mcdermott wrote:
> Hello, I was using tarlz 0.22 / liblz 1.13 like this:
>
>      tarlz -cpf file.tlz dir

Note that option '-p, --preserve-permissions' (don't subtract the umask on 
extraction) does not have effect on archive creation.

> to compress a 37GB directory 'dir' containing approximately 12,000 files.
> The output grew to 15GB and then I received this report:
>
>      File 'dir/foo' ends unexpectedly at pos 106496
>
> with exit code 1.  The file it errored on was an .exo cache file from the
> YouTube app (it was an sdcard backup of a phone), with size 1265209. It
> worked fine when I gave it just this file.  The file was not sparse when
> checked with 'ls -s'.

Do you mean the command 'tarlz -cpf file.tlz dir/foo' produced an archive 
with a 'dir/foo' member of the right size? Or did you use some other command?

> I looked at source code and this appears to be from minilzip.c in
> do_decompress(), so I am puzzled why this routine is used with '-c'
> option.

The diagnostic above does not come from minilzip.c, but from line 476 in 
function 'cworker' in create_lz.cc. It happens when the file being read and 
archived ends before its expected size. The reading is performad by the 
function 'readblock', which reads in a loop until the file ends or until 
something fails. So it is probable that the file 'dir/foo' has indeed some 
issue.

> I am also curious if there are any known bugs which might cause this
> error message to occur. The files were not in use at time of 'tarlz', as
> this was an old backup tree of a long-dead phone sdcard (stored on an
> unrelated ext4 fs).

This error message may happen when the file reduces its size while it is 
being archived, or when the file size does not match the real file contents 
because of a file system error, for example.

I have just added to my list of pending tasks to improve the diagnostic 
above to specify whether the cause is EOF or some error.

Best regards,
Antonio.