Re: [Qemu-block] [PATCH V4 01/10] specs/qcow2: add compress format extension

[Peter Lieven]

Am 20.07.2017 um 20:31 schrieb Eric Blake:

On 07/20/2017 11:26 AM, Peter Lieven wrote:

I haven't yet checked your code implementation to see where you are
setting window sizes, to know if window size is something that should be
a tunable in the file format.

You don't have to know the window size that was used. The docs of zlib

just say, that the windowSize specified at inflate must be greater or equal

to the one used at deflate. So if we change the code to use 15 we can safely

decompress old clusters that where compressed with 12.

Then you DO have to know the window size.  If I write my own qcow2
parser, and don't know what size you picked on compression, then how do
I know that the size I pick is >= your size during my decompression?

Even if you DON'T want size to be a tunable (and I'm on the fence on
that one), you STILL need the spec to state that for maximum
compatibility, when the incompatible feature is off, implementations
MUST use window size 12 for compression; and that when the compression
extension is used, implementations MUST use window size 15.

Or, by having window size be a tunable in the extension header allows
implementations to record window size, then you allow implemenations to
explicitly KNOW what window size they must meet or exceed.

But they still have to now what values to use if the header is absent.

Bottom line, you are right. Its cleaner to have that value in the header and

default it to zlib, level 0, windowBits 12 if the header is absent and on the other

side don't write the header if these values are used.

I checked the spec again expect for the windowBits nothing else is passed as

a parameter to inflateinit2.

Here is what is written about the windowBits provided to inflateInit2:

" The windowBits parameter is the base two logarithm of the maximum window size (the size of the history buffer). It should be in the range 8..15 for this version of the library. The default value is 15 if inflateInit is used instead. windowBits must be greater than or equal to the windowBitsvalue provided to deflateInit2() while compressing, or it must be equal to 15 if deflateInit2() was not used. If a compressed stream with a larger window size is given as input, inflate() will return with the error code Z_DATA_ERROR instead of trying to allocate a larger window."

I checked through the parameters of some compression algorithms. Most obviously have a positive interger for a compression level. And at least some have not a window as zlib, but a dictionary size which is also a power of two. So it might be reasonable to have that second

uint8_t also in the default header for a window/dictionary size exponent.

So it would propose to update the header like this:

    Byte  0 - 13:  compress_format_name
              14:  compress_level (uint8_t)
              15:  compress_window_exp (uint8_t)


Peter