[Qemu-devel] [PATCH v5 00/16] VHDX log replay and write support, .bdrv_create()

[Jeff Cody]

This patch series contains the initial VHDX log parsing, replay,
and write support.  (New with v4: VHDX image file creation)

=== v5 changes ===

v5 is also available for testing from:
https://github.com/codyprime/qemu-kvm-jtc/tree/vhdx-write-v5-upstream

Most of the patches from v4 -> v5 are the same, but there are a few differences
and a few new patches.  Here is a summary of which patches are different and/or
new:

$ ~/work/github/git-scripts/git-series-diff -u vhdx-write-v4-upstream -r 
qemu/master..HEAD
Key:
[----] : patches are identical
[####] : number of functional differences between patches in -u and -r series
[ new] : patch is new in the range given by -r
The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively

001/16:[----] [--] 'block: vhdx - minor comments and typo correction.'
002/16:[----] [--] 'block: vhdx - add header update capability.'
003/16:[----] [--] 'block: vhdx code movement - VHDXMetadataEntries and 
BDRVVHDXState to header.'
004/16:[----] [--] 'block: vhdx - log support struct and defines'
005/16:[----] [--] 'block: vhdx - break endian translation functions out'
006/16:[----] [--] 'block: vhdx - update log guid in header, and first write 
tracker'
007/16:[ new] 'block: vhdx code movement - move vhdx_close() above vhdx_open()'
008/16:[0070] [FC] 'block: vhdx - log parsing, replay, and flush support'
009/16:[ new] 'block: vhdx - add region overlap detection for image files'
010/16:[0003] [FC] 'block: vhdx - add log write support'
011/16:[----] [--] 'block: vhdx write support'
012/16:[ new] 'block: vhdx - remove BAT file offset bit shifting'
013/16:[----] [-C] 'block: vhdx - move more endian translations to 
vhdx-endian.c'
014/16:[----] [-C] 'block: vhdx - break out code operations to functions'
015/16:[----] [--] 'block: vhdx - fix comment typos in header, fix incorrect 
struct fields'
016/16:[----] [-C] 'block: vhdx - add .bdrv_create() support'


Patch highlights:

Patch 7  just some minor code movement, in prep for changes in patch 8

Patch 8  incorporates review feedback from Stefan, for the previous Patch 7
         in v4.

Patch 9  adds region checking for log, region table, and metadata tables, per
         suggestion from Stefan.

Patch 10 minor change from changes made in 8/16 (vhdx_guid_is_zero() is gone)

Patch 12 is just some minor housekeeping, to get rid of bit shifting that
         doesn't need to happen.



=== v4 changes ===  

v4 patches are available from github as well, on branch vhdx-write-v4-upstream:
https://github.com/codyprime/qemu-kvm-jtc/tree/vhdx-write-v4-upstream
https://github.com/codyprime/qemu-kvm-jtc.git

Those in the midst of reviewing v3, don't fear - the only changes with v4 is
the addition of patches on the end of the series (patches 10-13).  These
patches enable creating VHDX images.  Image files created have been
(briefly & lightly) tested on Hyper-V running on Windows Server 2012.

Some of the new patches could be squashed with earlier patches in the series,
but I refrained from doing so, since some of the patches have already been
reviewed, and others are in the midst of review.  I want to make it as easy
as possible on those currently reviewing. There is nothing critical
that needs to be pushed into the earlier patches.

New patches:

Patch 10:  Breaks out some more endian translation functions
            (likely squashable into patch 5)

Patch 11:  Break out some operations into seperate helper functions

Patch 12:  More comment typos and header fixes in vhdx.h
            (likely squashable into patch 1)

Patch 13:  Adds .bdrv_create() for vhdx.  VHDX images are can be created for
           Fixed or Dynamic images.

Patches 1-9 are unchanged.

=== end v4 changelog ===

=== v3 changes ===  

Thank you Kevin & Stefan for the feedback; incoporated in v3:

Patch 1: --- nil ---

Patch 2: * use sizeof(crc) instead of 4
         * remove outdated comment
         * use sizeof(MSGUID) instead of 16
         * direct assignment of guid structs rather than memcpy
         * rename 'rw' to 'generate_data_write_guid'
         * use offsetof() instead of 4
         * comment typos
         * add missing error checking
         * MSGUID is now QEMU_PACKED
         * configure enable for vhdx is now correct and not braindead

Patch 3: --- nil ---

Patch 4: * code style fixes
         * removed unused struct (VHDXLogEntryInfo)

Patch 5: * more direct assignment of guid rather than memcpy
         * order of operation in export/import the same now
         * became less generous with newlines (bah-humbug!)

Patch 6: * more direct assignment of guid rather than memcpy     
         * add error check in vhdx_user_visible_write(), now returns int

Patch 7: * check error return now of vhdx_user_visible_write()

Patch 8: * check error return now of vhdx_user_visible_write()
         * vhdx_log_write_sectors() uses bdrv_pwrite() vs bdrv_pwrite_sync()
         * more direct assignment of guid rather than memcpy     
         * use offsetof() instead of 4

Patch 9: --- nil ---

=== end v3 changelog ===

v2 changes:  Incorporated Fam's review feedback - Thank you Fam for the feedback

=== end v2 changelog ===

This will allow an existing log in a VHDX image to be replayed (e.g., a VHDX
image from a Hyper-V host that crashed).  In addition, metadata writes are
enabled through the log.  This allows write support to be enabled for VHDX,
as the BAT can be updated safely via the log journal.

These exact patches are available from github, for testing:
https://github.com/codyprime/qemu-kvm-jtc/tree/vhdx-write-v2-upstream

The latest vhdx work (including anything beyond these patches, such as
backing/parent file support) can be found at:
https://github.com/codyprime/qemu-kvm-jtc/tree/jtc-vhdx-latest

Jeff Cody (16):
  block: vhdx - minor comments and typo correction.
  block: vhdx - add header update capability.
  block: vhdx code movement - VHDXMetadataEntries and BDRVVHDXState to
    header.
  block: vhdx - log support struct and defines
  block: vhdx - break endian translation functions out
  block: vhdx - update log guid in header, and first write tracker
  block: vhdx code movement - move vhdx_close() above vhdx_open()
  block: vhdx - log parsing, replay, and flush support
  block: vhdx - add region overlap detection for image files
  block: vhdx - add log write support
  block: vhdx write support
  block: vhdx - remove BAT file offset bit shifting
  block: vhdx - move more endian translations to vhdx-endian.c
  block: vhdx - break out code operations to functions
  block: vhdx - fix comment typos in header, fix incorrect struct fields
  block: vhdx - add .bdrv_create() support

 block/Makefile.objs |    2 +-
 block/vhdx-endian.c |  216 ++++++++++
 block/vhdx-log.c    |  999 +++++++++++++++++++++++++++++++++++++++++++++
 block/vhdx.c        | 1134 ++++++++++++++++++++++++++++++++++++++++++++-------
 block/vhdx.h        |  176 ++++++--
 configure           |   24 ++
 6 files changed, 2375 insertions(+), 176 deletions(-)
 create mode 100644 block/vhdx-endian.c
 create mode 100644 block/vhdx-log.c

-- 
1.8.1.4