[Qemu-block] [PATCH RESEND 00/17] block: transactionless incremental backup series

[John Snow]

New topic, new version, same great incremental backup taste.

This series relies on [Qemu-devel] [PULL 30/69] blkdebug: fix "once" rule,
part of Stefan's 69 patch pull request submitted 2015-02-27.

This patchset enables the in-memory part of the incremental backup
feature, without transactional support.

Support for transactions will come at a later date, but getting this
portion upstream will help stabilize work on bitmap persistence and
bitmap migration.

Transactional support is being postponed to allow more development
to create a transactional callback system that will allow objects used
by the drive backup routines to perform cleanup actions conditionally
based on the outcome of all backups in the transaction group.

This series was once said to have been written by a man known only
to Qemu-Devel as "Fam Zheng", A programming maverick hellbent on never
again wasting space by making full backups when he did not have to. The
true origins of this series are now lost to the sands of time.

(11+ revisions later: Thanks, Fam!)

[----] : patches are identical
[####] : number of functional differences between upstream/downstream patch
[down] : patch is downstream-only
The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively

001/17:[0210] [FC] 'docs: incremental backup documentation'
002/17:[0002] [FC] 'qapi: Add optional field "name" to block dirty bitmap'
003/17:[----] [--] 'qmp: Ensure consistent granularity type'
004/17:[0027] [FC] 'qmp: Add block-dirty-bitmap-add and 
block-dirty-bitmap-remove'
005/17:[----] [--] 'block: Introduce bdrv_dirty_bitmap_granularity()'
006/17:[----] [--] 'hbitmap: add hbitmap_merge'
007/17:[down] 'block: Add bitmap disabled status'
008/17:[0017] [FC] 'block: Add bitmap successors'
009/17:[0004] [FC] 'qmp: Add support of "dirty-bitmap" sync mode for 
drive-backup'
010/17:[0011] [FC] 'qmp: add block-dirty-bitmap-clear'
011/17:[0004] [FC] 'qmp: Add dirty bitmap status fields in query-block'
012/17:[----] [--] 'block: add BdrvDirtyBitmap documentation'
013/17:[----] [--] 'block: Ensure consistent bitmap function prototypes'
014/17:[0019] [FC] 'block: Resize bitmaps on bdrv_truncate'
015/17:[0002] [FC] 'iotests: add invalid input incremental backup tests'
016/17:[----] [--] 'iotests: add simple incremental backup case'
017/17:[0008] [FC] 'iotests: add incremental backup failure recovery test'

Deletions:
 - Removed Transactions, to be added later.
 - Removed Transaction tests, as above.

Changes:
01: Indentation fixes.
    Removed enable/disable documentation.
    Added a note that transactions aren't implemented yet.
    Removed my needless commas
    Added error case documentation.

07: QMP enable/disable commands are deleted.

14: Some comments concerning assertions.
    Scrub re-alloc memory if we expand the array.
    Do not attempt to scrub memory if fix_count is 0

Changes made with Reviews kept:

02: Since 2.4
04: Since 2.4
    Demingled the QMP command documentation.
08: Additions to what was qmp_block_dirty_enable/disable
    are no longer present as those function no longer exist.
09: Since 2.4
10: Since 2.4
    Demingled QMP command documentation.
11: Since 2.4
15: Test 112 --> 124
17: Number of tests altered. (Only 4, now.)

Fam Zheng (1):
  qapi: Add optional field "name" to block dirty bitmap

John Snow (16):
  docs: incremental backup documentation
  qmp: Ensure consistent granularity type
  qmp: Add block-dirty-bitmap-add and block-dirty-bitmap-remove
  block: Introduce bdrv_dirty_bitmap_granularity()
  hbitmap: add hbitmap_merge
  block: Add bitmap disabled status
  block: Add bitmap successors
  qmp: Add support of "dirty-bitmap" sync mode for drive-backup
  qmp: add block-dirty-bitmap-clear
  qmp: Add dirty bitmap status fields in query-block
  block: add BdrvDirtyBitmap documentation
  block: Ensure consistent bitmap function prototypes
  block: Resize bitmaps on bdrv_truncate
  iotests: add invalid input incremental backup tests
  iotests: add simple incremental backup case
  iotests: add incremental backup failure recovery test

 block.c                       | 248 ++++++++++++++++++++++++++++++++--
 block/backup.c                | 149 +++++++++++++++++----
 block/mirror.c                |  46 +++----
 blockdev.c                    | 162 +++++++++++++++++++++-
 docs/bitmaps.md               | 303 ++++++++++++++++++++++++++++++++++++++++++
 hmp.c                         |   3 +-
 include/block/block.h         |  34 ++++-
 include/block/block_int.h     |   4 +-
 include/qemu/hbitmap.h        |  21 +++
 migration/block.c             |   9 +-
 qapi/block-core.json          |  92 ++++++++++++-
 qmp-commands.hx               |  92 ++++++++++++-
 tests/qemu-iotests/124        | 266 ++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/124.out    |   5 +
 tests/qemu-iotests/group      |   1 +
 tests/qemu-iotests/iotests.py |   6 +-
 util/hbitmap.c                |  86 ++++++++++++
 17 files changed, 1437 insertions(+), 90 deletions(-)
 create mode 100644 docs/bitmaps.md
 create mode 100644 tests/qemu-iotests/124
 create mode 100644 tests/qemu-iotests/124.out

-- 
1.9.3