Re: [Qemu-devel] Some uncertain about implementing stream optimized writing

[Fam Zheng]

On Wed, Jun 29, 2011 at 4:15 PM, Kevin Wolf <address@hidden> wrote:
> Am 29.06.2011 06:47, schrieb Fam Zheng:
>> Hi,
>>
>> I have implemented reading for sparse optimized and come to implement
>> writing. It is a little complicated and I am not sure what is the best
>> approach. Could you give me some advice?
>>
>> Here is the details: (pasted from http://warm.la/soc/?p=98)
>>
>> Stream optimized VMDK image allocates minimized space for a compressed
>> cluster, which means if there is high compress ratio, a cluster
>> possibly only takes one physical sector in the file. It makes
>> overwriting hard, especially when new data needs more sectors than
>> previously allocated.
>>
>> Attach the image to VMware and boot the VM to test this format, it
>> seems that VMware wouldn’t do write to allocated clusters, but can
>> allocate new cluster to save data.
>
> I would have expected that they copy the data to a new cluster, like
> qcow2 does.
>
> Of course, it doesn't make a lot of sense to do COW if you can't free
> the old space. Doing qcow2 style compression (which it seems to be from
> your description) without having reference counts seems a bit stupid...
>
> If you really can't write to already allocated clusters in VMware,
> what's the use case for this? Have one read-only partition and
> everything else free?

I think it's only for improving transmission efficiency, as the name
"stream optimized" indicates.

>
>> Overwriting existing cluster requires measuring new data size and
>> deciding whether in-place overwrite is OK, if not we must look for
>> other free space. Metadata in image has no such information for sector
>> allocation algorithm, so if we want to fully enable writing to stream
>> optimized, sector allocation bitmap will be introduced into block
>> state. This should significantly increase memory usage and somehow
>> complicate the driver.
>
> That would be 256k per GB. Sounds like it could become very large
> indeed. Maybe better a free list approach and an image scan, like what
> was considered for QED. Such a scan is unacceptable for a native format,
> but maybe good enough for VMDK.
>
> But how much is this format really used? Is it worth the trouble? If it
> doesn't exist in practice, let's just leak the clusters. Whoever needs
> more disk space should use qemu-img convert to compact it again.
>
It does exist and vmware-vdiskmanager produces such format. But I
don't think in practice anyone would try to run a VM on it.


-- 
Best regards!
Fam Zheng