Re: [Qemu-devel] [RFC] qed: Add QEMU Enhanced Disk format

[Anthony Liguori]

On 09/07/2010 09:51 AM, Avi Kivity wrote:

I'll let Stefan address most of this.

> >      uint32_t first_cluster;       /* in clusters */
>
> First cluster of what?

This should probably be header_size /* in clusters */ because that's 
what it really means.

> Need a checksum for the header.

Is that not a bit overkill for what we're doing?  What's the benefit?

> > The L2 link '''should''' be made after the data is in place on 
> > storage.  However, when no ordering is enforced the worst case 
> > scenario is an L2 link to an unwritten cluster.
>
> Or it may cause corruption if the physical file size is not committed, 
> and L2 now points at a free cluster.

An fsync() will make sure the physical file size is committed.  The 
metadata does not carry an additional integrity guarantees over the 
actual disk data except that in order to avoid internal corruption, we 
have to order the L2 and L1 writes.

As part of the read process, it's important to validate that the L2 
entries don't point to blocks beyond EOF.  This is an indication of a 
corrupted I/O operation and we need to treat that as an unallocated cluster.

> > The L1 link '''must''' be made after the L2 cluster is in place on 
> > storage.  If the order is reversed then the L1 table may point to a 
> > bogus L2 table.  (Is this a problem since clusters are allocated at 
> > the end of the file?)
> >
> > ==Grow==
> > # If table_size * TABLE_NOFFSETS < new_image_size, fail -EOVERFLOW.  
> > The L1 table is not big enough.
>
> With a variable-height tree, we allocate a new root, link its first 
> entry to the old root, and write the new header with updated root and 
> height.
>
> > # Write new image_size header field.
> >
> > =Data integrity=
> > ==Write==
> > Writes that complete before a flush must be stable when the flush 
> > completes.
> >
> > If storage is interrupted (e.g. power outage) then writes in progress 
> > may be lost, stable, or partially completed.  The storage must not be 
> > otherwise corrupted or inaccessible after it is restarted.
>
> We can remove this requirement by copying-on-write any metadata write, 
> and keeping two copies of the header (with version numbers and checksums).

QED has a property today that all metadata or cluster locations have a 
single location on the disk format that is immutable.  Defrag would 
relax this but defrag can be slow.

Having an immutable on-disk location is a powerful property which 
eliminates a lot of complexity with respect to reference counting and 
dealing with free lists.

For the initial design I would avoid introducing something like this.  
One of the nice things about features is that we can introduce 
multi-level trees as a future feature if we really think it's the right 
thing to do.

But we should start at a simple design with high confidence and high 
performance, and then introduce features with the burden that we're 
absolutely sure that we don't regress integrity or performance.

Regards,

Anthony Liguori

>   Enterprise storage will not corrupt on writes, but commodity storage 
> may.