[Qemu-devel] Re: [PATCH v2 3/7] docs: Add QED image format specification

[Anthony Liguori]

On 10/11/2010 10:24 AM, Avi Kivity wrote:
>  On 10/11/2010 05:02 PM, Anthony Liguori wrote:
> > On 10/11/2010 08:44 AM, Avi Kivity wrote:
> > >  On 10/11/2010 03:42 PM, Stefan Hajnoczi wrote:
> > > > >
> > > > >  A leak is acceptable (it won't grow; it's just an unused, incorrect
> > > > >  freelist), but data corruption is not.
> > > >
> > > > The alternative is for the freelist to be a non-compat feature bit.
> > > > That means older QEMU binaries cannot use a QED image that has enabled
> > > > the freelist.
> > >
> > > For this one feature.  What about others?
> >
> > A compat feature is one where the feature can be completely ignored 
> > (meaning that the QEMU does not have to understand the data format).
> >
> > An example of a compat feature is copy-on-read.  It's merely a 
> > suggestion and there is no additional metadata.  If a QEMU doesn't 
> > understand it, it doesn't affect it's ability to read the image.
> >
> > An example of a non-compat feature would be zero cluster entries.  
> > Zero cluster entries are a special L2 table entry that indicates that 
> > a cluster's on-disk data is all zeros.  As long as there is at least 
> > 1 ZCE in the L2 tables, this feature bit must be set.  As soon as all 
> > of the ZCE bits are cleared, the feature bit can be unset.
> >
> > An older QEMU will gracefully fail when presented with an image using 
> > ZCE bits.  An image with no ZCEs will work on older QEMUs.
>
> What's the motivation behind ZCE?

It's very useful for Copy-on-Read.  If the cluster in the backing file 
is unallocated, then when you do a copy-on-read, you don't want to write 
out a zero cluster since you'd expand the image to it's maximum size.

It's also useful for operations like compaction in the absence of TRIM.  
The common implementation on platforms like VMware is to open a file and 
write zeros to it until it fills up the filesystem.  You then delete the 
file.  The result is that any unallocated data on the disk is written as 
zero and combined with zero-detection in the image format, you can 
compact the image size by marking unallocated blocks as ZCE.

> There is yet a third type of feature, one which is not strictly needed 
> in order to use the image, but if used, must be kept synchronized.  An 
> example is the freelist.  Another example is a directory index for a 
> filesystem.  I can't think of another example which would be relevant 
> to QED -- metadata checksums perhaps? -- we can always declare it a 
> non-compatible feature, but of course, it reduces compatibility.

You're suggesting a feature that is not strictly needed, but that needs 
to be kept up to date.  If it can't be kept up to date, something needs 
to happen to remove it.  Let's call this a transient feature.

Most of the transient features can be removed given some bit of code.  
For instance, ZCE can be removed by writing out zero clusters or writing 
an unallocated L2 entry if there is no backing file.

I think we could add a qemu-img demote command or something like that 
that attempted to remove features when possible.  That doesn't give you 
instant compatibility but I'm doubtful that you can come up with a 
generic way to remove a feature from an image without knowing anything 
about the image.

Regards,

Anthony Liguori

>