Re: plzip: manual gives very false numbers, real defaults are huge!

[Antonio Diaz Diaz]

Steffen Nurpmeso wrote:
> Thanks for the quick response on a Saturday.

You are welcome. :-)

>   Note that the number of usable threads is limited by file size;
>   on files larger than  a  few  GB plzip  can  use  hundreds  of
>   processors, but on files of only a few MB plzip is no faster than lzip.
>
> Ok "you get scaling effects", but 70 MiB is not "a few MiB".

The above means "on files of only a few MB plzip can't be faster than lzip, 
no matter what options you use". Of course, at high compression levels the 
"few MB" become "several tens of MB".

> 67 megabytes per processor!  (How about doing a stat and somehow taking
> into account st_size?  Or fstat, after the file was opened?

This would break reproducibility (obtaining identical compressed output from 
identical input) because the size of uncompressed data read from standard 
input (not from a file) can't be known in advance.

> A single sentence that the "defaults" are (of course?!!?!)
> dependent on the compression level would have shed some enlightening.

I'll try to document it better in the manual and in the man page.

> (Having read the referenced section in the .info file in the
> source tarball i would open an issue as "wishlist" asking for an
> option that would scale-to-"a-reasonable-number-of"-cpus.

As I said above, such an option would not work with data read from standard 
input, and would break reproducibility.

Best regards,
Antonio.