Re: [Qemu-devel] Some uncertain about implementing stream optimized writing

[Kevin Wolf]

Am 29.06.2011 06:47, schrieb Fam Zheng:
> Hi,
> 
> I have implemented reading for sparse optimized and come to implement
> writing. It is a little complicated and I am not sure what is the best
> approach. Could you give me some advice?
> 
> Here is the details: (pasted from http://warm.la/soc/?p=98)
> 
> Stream optimized VMDK image allocates minimized space for a compressed
> cluster, which means if there is high compress ratio, a cluster
> possibly only takes one physical sector in the file. It makes
> overwriting hard, especially when new data needs more sectors than
> previously allocated.
> 
> Attach the image to VMware and boot the VM to test this format, it
> seems that VMware wouldn’t do write to allocated clusters, but can
> allocate new cluster to save data.

I would have expected that they copy the data to a new cluster, like
qcow2 does.

Of course, it doesn't make a lot of sense to do COW if you can't free
the old space. Doing qcow2 style compression (which it seems to be from
your description) without having reference counts seems a bit stupid...

If you really can't write to already allocated clusters in VMware,
what's the use case for this? Have one read-only partition and
everything else free?

> Overwriting existing cluster requires measuring new data size and
> deciding whether in-place overwrite is OK, if not we must look for
> other free space. Metadata in image has no such information for sector
> allocation algorithm, so if we want to fully enable writing to stream
> optimized, sector allocation bitmap will be introduced into block
> state. This should significantly increase memory usage and somehow
> complicate the driver.

That would be 256k per GB. Sounds like it could become very large
indeed. Maybe better a free list approach and an image scan, like what
was considered for QED. Such a scan is unacceptable for a native format,
but maybe good enough for VMDK.

But how much is this format really used? Is it worth the trouble? If it
doesn't exist in practice, let's just leak the clusters. Whoever needs
more disk space should use qemu-img convert to compact it again.

Kevin