Re: zlib-ng as a compat replacement for zlib

[Richard W.M. Jones]

On Fri, Sep 01, 2023 at 06:45:22PM +0200, Florian Weimer wrote:
> * Richard W. M. Jones:
> 
> > I tested the speed of decompression using:
> >
> >   $ hyperfine 'qemu-img convert -W -m 16 -f qcow2 test.qcow2.XXX -O raw 
> > test.out'
> >   (qemu 8.0.0-4.fc39.x86_64)
> >
> >   $ hyperfine 'nbdkit -U - --filter=qcow2dec file test.qcow2.XXX --run 
> > '\''nbdcopy --request-size "$uri" test.out'\'' '
> >   (nbdkit-1.35.11-2.fc40.x86_64)
> 
> How realistic is that?  Larger cluster sizes will make random access
> perform noticeably worse is some cases.  Think about reading a few bytes
> towards the end of the cluster.  It makes a difference whether you have
> to decompress 64 KiB bytes for that, or 2 MiB.  As far as I understand
> it, the above commands use all data decompressed, so they don't suffer
> from this issue (particularly with read-ahead to deal with unfortunate
> cluster boundaries).
> 
> Time to first HTTP request served after boot or something like that
> might be a better comparison.

Yes, this is a good point.

Current Fedora images use 64k cluster size which I think is also the
default:

$ qemu-img info 
https://download.fedoraproject.org/pub/fedora/linux/releases/38/Cloud/x86_64/images/Fedora-Cloud-Base-38-1.6.x86_64.qcow2
 
image: 
https://download.fedoraproject.org/pub/fedora/linux/releases/38/Cloud/x86_64/images/Fedora-Cloud-Base-38-1.6.x86_64.qcow2
file format: qcow2
virtual size: 5 GiB (5368709120 bytes)
disk size: unavailable
cluster_size: 65536                    <----
Format specific information:
    compat: 0.10
    compression type: zlib
    refcount bits: 16

And the tests showed there's barely any performance gain above the
default cluster size anyway.  Only very small clusters should be
avoided, but there are good reasons to avoid those already such as
metadata overhead and small maximum image size.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top