Re: [Qemu-devel] Re: [PATCH v4 1/5] docs: Add QED image format specification

[Stefan Hajnoczi]

On Fri, Nov 12, 2010 at 1:58 PM, Kevin Wolf <address@hidden> wrote:
> Am 28.10.2010 13:01, schrieb Stefan Hajnoczi:
>> Signed-off-by: Stefan Hajnoczi <address@hidden>
>> ---
>>  docs/specs/qed_spec.txt |  128 
>> +++++++++++++++++++++++++++++++++++++++++++++++
>>  1 files changed, 128 insertions(+), 0 deletions(-)
>>  create mode 100644 docs/specs/qed_spec.txt
>>
>> diff --git a/docs/specs/qed_spec.txt b/docs/specs/qed_spec.txt
>> new file mode 100644
>> index 0000000..e4425c8
>> --- /dev/null
>> +++ b/docs/specs/qed_spec.txt
>> @@ -0,0 +1,128 @@
>> +=Specification=
>> +
>> +The file format looks like this:
>> +
>> + +----------+----------+----------+-----+
>> + | cluster0 | cluster1 | cluster2 | ... |
>> + +----------+----------+----------+-----+
>> +
>> +The first cluster begins with the '''header'''.  The header contains 
>> information about where regular clusters start; this allows the header to be 
>> extensible and store extra information about the image file.  A regular 
>> cluster may be a '''data cluster''', an '''L2''', or an '''L1 table'''.  L1 
>> and L2 tables are composed of one or more contiguous clusters.
>> +
>> +Normally the file size will be a multiple of the cluster size.  If the file 
>> size is not a multiple, extra information after the last cluster may not be 
>> preserved if data is written.  Legitimate extra information should use space 
>> between the header and the first regular cluster.
>> +
>> +All fields are little-endian.
>> +
>> +==Header==
>> + Header {
>> +     uint32_t magic;               /* QED\0 */
>> +
>> +     uint32_t cluster_size;        /* in bytes */
>> +     uint32_t table_size;          /* for L1 and L2 tables, in clusters */
>> +     uint32_t header_size;         /* in clusters */
>> +
>> +     uint64_t features;            /* format feature bits */
>> +     uint64_t compat_features;     /* compat feature bits */
>> +     uint64_t l1_table_offset;     /* in bytes */
>> +     uint64_t image_size;          /* total logical image size, in bytes */
>> +
>> +     /* if (features & QED_F_BACKING_FILE) */
>> +     uint32_t backing_filename_offset; /* in bytes from start of header */
>> +     uint32_t backing_filename_size;   /* in bytes */
>> + }
>> +
>> +Field descriptions:
>> +* ''cluster_size'' must be a power of 2 in range [2^12, 2^26].
>> +* ''table_size'' must be a power of 2 in range [1, 16].
>> +* ''header_size'' is the number of clusters used by the header and any 
>> additional information stored before regular clusters.
>> +* ''features'', ''compat_features'', and ''autoclear_features'' are file 
>> format extension bitmaps.  They work as follows:
>> +** An image with unknown ''features'' bits enabled must not be opened.  
>> File format changes that are not backwards-compatible must use ''features'' 
>> bits.
>> +** An image with unknown ''compat_features'' bits enabled can be opened 
>> safely.  The unknown features are simply ignored and represent 
>> backwards-compatible changes to the file format.
>> +** An image with unknown ''autoclear_features'' bits enable can be opened 
>> safely after clearing the unknown bits.  This allows for 
>> backwards-compatible changes to the file format which degrade gracefully and 
>> can be re-enabled again by a new program later.
>
> autoclear features aren't even part of the header in the spec.

Thanks for spotting this, I forgot to sync the header with the code.

>> +* ''l1_table_offset'' is the offset of the first byte of the L1 table in 
>> the image file and must be a multiple of ''cluster_size''.
>> +* ''image_size'' is the block device size seen by the guest and must be a 
>> multiple of 512 bytes.
>> +* ''backing_filename'' is a string in (byte offset, byte size) form.  It is 
>> not NUL-terminated and has no alignment constraints.
>> +
>> +Feature bits:
>> +* QED_F_BACKING_FILE = 0x01.  The image uses a backing file.  The backing 
>> filename string is given in the ''backing_filename_{offset,size}'' fields 
>> and may be an absolute path or relative to the image file.
>> +* QED_F_NEED_CHECK = 0x02.  The image needs a consistency check before use.
>> +* QED_F_BACKING_FORMAT_NO_PROBE = 0x04.  The backing file is a raw disk 
>> image and no file format autodetection should be attempted.  This should be 
>> used to ensure that raw backing images are never detected as an image format 
>> if they happen to contain magic constants.
>> +
>> +There are currently no defined ''compat_features'' or 
>> ''autoclear_features'' bits.
>> +
>> +Fields predicated on a feature bit are only used when that feature is set.  
>> The fields always take up header space, regardless of whether or not the 
>> feature bit is set.
>> +
>> +==Tables==
>> +
>> +Tables provide the translation from logical offsets in the block device to 
>> cluster offsets in the file.
>> +
>> + #define TABLE_NOFFSETS (table_size * cluster_size / sizeof(uint64_t))
>> +
>> + Table {
>> +     uint64_t offsets[TABLE_NOFFSETS];
>> + }
>> +
>> +The tables are organized as follows:
>> +
>> +                    +----------+
>> +                    | L1 table |
>> +                    +----------+
>> +               ,------'  |  '------.
>> +          +----------+   |    +----------+
>> +          | L2 table |  ...   | L2 table |
>> +          +----------+        +----------+
>> +      ,------'  |  '------.
>> + +----------+   |    +----------+
>> + |   Data   |  ...   |   Data   |
>> + +----------+        +----------+
>> +
>> +A table is made up of one or more contiguous clusters.  The table_size 
>> header field determines table size for an image file.  For example, 
>> cluster_size=64 KB and table_size=4 results in 256 KB tables.
>> +
>> +The logical image size must be less than or equal to the maximum possible 
>> size of clusters rooted by the L1 table:
>> + header.image_size <= TABLE_NOFFSETS * TABLE_NOFFSETS * header.cluster_size
>> +
>> +All offsets in L1 and L2 tables are cluster-aligned.  The least significant 
>> bits up to ''cluster_size'' are reserved and must be zero.
>
> I know what you mean here, but the text leaves things a bit unclear.
> First I would expect a bit number instead of a byte offset for "bits up
> to x". Second, cluster_size is the first bit not reserved, whereas your
> description sounds to me as if it included cluster_size.

Will fix.

Stefan