Re: zlib-ng as a compat replacement for zlib

[Kevin Wolf]

Am 31.08.2023 um 11:20 hat Richard W.M. Jones geschrieben:
> On Thu, Aug 31, 2023 at 11:05:55AM +0200, Kevin Wolf wrote:
> > [ Cc: qemu-block ]
> > 
> > Am 30.08.2023 um 20:26 hat Richard W.M. Jones geschrieben:
> > > On Tue, Aug 29, 2023 at 05:49:24PM -0000, Daniel Alley wrote:
> > > > > The background to this is I've spent far too long trying to optimize
> > > > > the conversion of qcow2 files to raw files.  Most existing qcow2 files
> > > > > that you can find online are zlib compressed, including the qcow2
> > > > > images provided by Fedora.  Each cluster in the file is separately
> > > > > compressed as a zlib stream, and qemu uses zlib library functions to
> > > > > decompress them.  When downloading and decompressing these files, I
> > > > > measured 40%+ of the total CPU time is doing zlib decompression.
> > > > > 
> > > > > [You don't need to tell me how great Zstd is, qcow2 supports this for
> > > > > compression also, but it is not widely used by existing content.]
> > 
> > You make it sound like compressing each cluster individually has a big
> > impact. If so, does increasing the cluster size make a difference, too?
> > That could be an change with less compatibility concerns.
> 
> The issue we're discussing in the original thread is speed of
> decompression.  We noted that using zlib-ng (a not-quite drop-in
> replacement for zlib) improves decompression speed by 40% or more.
> 
> Original thread:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/CDNPJ4SOTRQMYVCDI3ZSY4SP4FYESCWD/
> zlib-ng proposed change:
> https://src.fedoraproject.org/rpms/zlib-ng/pull-request/3
> 
> Size of the compressed file is also a concern, but wasn't discussed.

I understand the context and didn't really think about file size at all.

My question was essentially if decompressing many small blocks (as we do
today) performs significantly different from decompressing fewer larger
blocks (as we would do with a larger cluster size).

Kevin