[Qemu-devel] [PATCH v4 00/38] blockdev: BlockBackend and media

[Max Reitz]

First of all: Thank you, Eric and Berto, for reviewing v3! And thank
you, Fam, for at least having a peek at it and being confident enough to
base a series of your own on it. :-)

This series reworks a lot regarding BlockBackend and media. Basically,
it allows empty BlockBackends, that is BBs without a BDS tree.

Before this series, empty drives are represented by a BlockBackend with
an empty BDS attached to it (a BDS with a NULL driver). However, now we
have BlockBackends, thus an empty drive should be represented by a
BlockBackend without any BDS tree attached to it. This is what this
series does.


Quick and early summary for the v4 changes:
- Rebase on master (most changes due to the new throttle groups)
- Addressed comments for v3
- Fixed a bug: Exchanging media should always be possible for BBs
  without an attached device model (it wasn't in v3)


Justification for each of the patches and their order:

-- Preparation before _is_inserted() patches --
 1: Patch 9 will not take care not to break host floppy support, so that
    support needs to be removed first.
 2: Needed for patch 3. Patch 24 is a follow-up after BDS-less BBs are
    allowed.
 3: bdrv_close_all() is broken ("block: Rework bdrv_close_all()"). Patch
    6 will break iotest 071 (actually, just make the problem apparent).
    So this patch is required to work around the issue.
    (with "the issue" being that bdrv_close_all() does not unref() the
    BDSs it is closing, but just force-closes everything, even if the
    BDS may still be in use somewhere)

-- _is_inserted() patches --
 4: General clean-up work, nice to have before patch 6 (and goes in tune
    with patch 5).
 5: Using the same BB as a guest device means that the data read from
    there should be exactly the same. Opening the guest tray should
    therefore result in no data being readable. This is what we then
    need this function for.
 6: General clean-up work (in the _is_inserted() area).
 7: General clean-up work (in the _is_inserted() area).
 8: General clean-up work (also regarding _is_inserted()).
 9: Required so inserting a floppy will not result in the tray being
    reported as closed (you need to "push in" the floppy first, using
    blockdev-close-tray). It's here in the "_is_inserted() patches area"
    because I feel like that's a very related topic.

-- Support for BDS-less BBs --
10: Preparation for BDS-less BBs
11: Preparation for BDS-less BBs
12: Preparation for BDS-less BBs (BB properties should be in the BB, and
    not in the root BDS)
13: Patch 14 removes BlockAcctStats from the BDS, but wr_highest_sector
    is BDS-dependent, so it needs to stay here
14: Preparation for BDS-less BBs (BB properties should be in the BB, and
    not in the root BDS)
15: Preparation for BDS-less BBs (BB properties should be in the BB, and
    not in the root BDS)
16: Preparation for BDS-less BBs (Removing a BDS tree should retain some
    properties for legacy reasons, which must therefore be stored in the
    BB(RS))
17: Preparation for BDS-less BBs
18: Preparation for BDS-less BBs
19: Preparation for BDS-less BBs
20: Ability to add BDS trees to empty BBs ("inserting a medium")
21: Preparation for BDS-less BBs (needs patch 20)
22: One goal of this series, and fixes the "opening tray" event for
    empty drives when shutting down qemu
23: Needed for patch 24
24: Completion of what patch 2 begun
25: Ability to detach BDS trees from BBs

-- "Atomic" QMP tray operations --
26: blockdev-open-tray
27: blockdev-close-tray
28: blockdev-remove-medium
29: blockdev-insert-medium

-- Reimplementation of change/eject --
30: eject
31: change
32: Clean-up patch

-- New QMP blockdev-change-medium command --
33: New QMP command
34: Use for HMP change command
35: Add flag to that command for changing the read-only access mode
    (which was my original intention for this series)
36: Same flag for HMP

-- Tests --
37: Required for patch 38
38: iotests are always nice, so here is one


v4:
- Rebased on current master:
  - Patch 1 (all in code being removed):
    - DEBUG_FLOPPY has been removed
    - floppy_probe_device() was a bit more specific
  - Patch 16 (because of throttle groups): Store the throttle group name
    in the BBRS
  - Patch 21: hmp_drive_del() no longer drains or flushes explicitly
  - Patch 22: Retain throttle group for BB-BDS trees, and save throttle
    group name in the BBRS for empty BBs
  - Patch 23:
    - Read the throttle group name in the new function
    - Use macros for "cache.*"
  - Patch 24: Set the throttle group for BB-less BDS trees
  - Patches 26, 27, 28, 29, 30, 31: Drop QERR_DEVICE_NOT_FOUND
  - Patch 31:
    - Use error_setg() for QERR_INVALID_BLOCK_FORMAT
    - Respect the throttle group name stored in the BBRS
- Patch 23: Fixed potentially unused variable [Fam], and removed the
  empty line at the function's end
- Patches 26, 27, 28, 29, 33: s/2\.3/2\.5/ [Eric]
- Patches 28, 29: Assume the tray to be open if no device model is
  attached to the BB; while blk_dev_has_removable_media() will actually
  return true in this configuration, blk_dev_is_tray_open() will return
  false. But without a device, there is noone stopping you from
  exchanging the medium, so we can just ignore that.


git-backport-diff against v3:

Key:
[----] : patches are identical
[####] : number of functional differences between upstream/downstream patch
[down] : patch is downstream-only
The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively

001/38:[0025] [FC] 'block: Remove host floppy support'
002/38:[----] [--] 'blockdev: Allow creation of BDS trees without BB'
003/38:[----] [--] 'iotests: Only create BB if necessary'
004/38:[----] [--] 'block: Make bdrv_is_inserted() return a bool'
005/38:[----] [--] 'block: Add blk_is_available()'
006/38:[----] [--] 'block: Make bdrv_is_inserted() recursive'
007/38:[----] [--] 'block/quorum: Implement bdrv_is_inserted()'
008/38:[----] [-C] 'block: Invoke change media CB before NULLing drv'
009/38:[----] [--] 'hw/block/fdc: Implement tray status'
010/38:[----] [--] 'hw/usb-storage: Check whether BB is inserted'
011/38:[----] [--] 'block: Fix BB AIOCB AioContext without BDS'
012/38:[----] [-C] 'block: Move guest_block_size into BlockBackend'
013/38:[----] [--] 'block: Remove wr_highest_sector from BlockAcctStats'
014/38:[----] [-C] 'block: Move BlockAcctStats into BlockBackend'
015/38:[----] [-C] 'block: Move I/O status and error actions into BB'
016/38:[0016] [FC] 'block: Add BlockBackendRootState'
017/38:[----] [--] 'block: Make some BB functions fall back to BBRS'
018/38:[----] [--] 'block: Fail requests to empty BlockBackend'
019/38:[----] [--] 'block: Prepare remaining BB functions for NULL BDS'
020/38:[----] [--] 'block: Add blk_insert_bs()'
021/38:[0002] [FC] 'block: Prepare for NULL BDS'
022/38:[0016] [FC] 'blockdev: Do not create BDS for empty drive'
023/38:[0028] [FC] 'blockdev: Pull out blockdev option extraction'
024/38:[0008] [FC] 'blockdev: Allow more options for BB-less BDS tree'
025/38:[----] [--] 'block: Add blk_remove_bs()'
026/38:[0005] [FC] 'blockdev: Add blockdev-open-tray'
027/38:[0005] [FC] 'blockdev: Add blockdev-close-tray'
028/38:[0013] [FC] 'blockdev: Add blockdev-remove-medium'
029/38:[0013] [FC] 'blockdev: Add blockdev-insert-medium'
030/38:[0003] [FC] 'blockdev: Implement eject with basic operations'
031/38:[0010] [FC] 'blockdev: Implement change with basic operations'
032/38:[----] [--] 'block: Inquire tray state before tray-moved events'
033/38:[0002] [FC] 'qmp: Introduce blockdev-change-medium'
034/38:[----] [--] 'hmp: Use blockdev-change-medium for change command'
035/38:[----] [-C] 'blockdev: read-only-mode for blockdev-change-medium'
036/38:[----] [-C] 'hmp: Add read-only-mode option to change command'
037/38:[----] [--] 'iotests: More options for VM.add_drive()'
038/38:[----] [-C] 'iotests: Add test for change-related QMP commands'


Max Reitz (38):
  block: Remove host floppy support
  blockdev: Allow creation of BDS trees without BB
  iotests: Only create BB if necessary
  block: Make bdrv_is_inserted() return a bool
  block: Add blk_is_available()
  block: Make bdrv_is_inserted() recursive
  block/quorum: Implement bdrv_is_inserted()
  block: Invoke change media CB before NULLing drv
  hw/block/fdc: Implement tray status
  hw/usb-storage: Check whether BB is inserted
  block: Fix BB AIOCB AioContext without BDS
  block: Move guest_block_size into BlockBackend
  block: Remove wr_highest_sector from BlockAcctStats
  block: Move BlockAcctStats into BlockBackend
  block: Move I/O status and error actions into BB
  block: Add BlockBackendRootState
  block: Make some BB functions fall back to BBRS
  block: Fail requests to empty BlockBackend
  block: Prepare remaining BB functions for NULL BDS
  block: Add blk_insert_bs()
  block: Prepare for NULL BDS
  blockdev: Do not create BDS for empty drive
  blockdev: Pull out blockdev option extraction
  blockdev: Allow more options for BB-less BDS tree
  block: Add blk_remove_bs()
  blockdev: Add blockdev-open-tray
  blockdev: Add blockdev-close-tray
  blockdev: Add blockdev-remove-medium
  blockdev: Add blockdev-insert-medium
  blockdev: Implement eject with basic operations
  blockdev: Implement change with basic operations
  block: Inquire tray state before tray-moved events
  qmp: Introduce blockdev-change-medium
  hmp: Use blockdev-change-medium for change command
  blockdev: read-only-mode for blockdev-change-medium
  hmp: Add read-only-mode option to change command
  iotests: More options for VM.add_drive()
  iotests: Add test for change-related QMP commands

 block.c                        | 169 +-------
 block/accounting.c             |   8 -
 block/backup.c                 |  17 +-
 block/block-backend.c          | 387 +++++++++++++++--
 block/commit.c                 |   3 +-
 block/io.c                     |  10 +-
 block/mirror.c                 |  17 +-
 block/qapi.c                   |  36 +-
 block/quorum.c                 |  16 +
 block/raw-posix.c              | 225 +---------
 block/raw_bsd.c                |   2 +-
 block/stream.c                 |   3 +-
 blockdev.c                     | 920 +++++++++++++++++++++++++++++------------
 blockjob.c                     |   5 +-
 hmp-commands.hx                |  20 +-
 hmp.c                          |  47 ++-
 hw/block/fdc.c                 |  20 +-
 hw/block/xen_disk.c            |   4 +-
 hw/usb/dev-storage.c           |  30 +-
 include/block/accounting.h     |   3 -
 include/block/block.h          |  16 +-
 include/block/block_int.h      |  25 +-
 include/qemu/typedefs.h        |   1 +
 include/sysemu/block-backend.h |  14 +-
 include/sysemu/blockdev.h      |   2 -
 migration/block.c              |   5 +
 monitor.c                      |   4 +
 qapi-schema.json               |   6 +-
 qapi/block-core.json           | 138 ++++++-
 qmp-commands.hx                | 213 +++++++++-
 qmp.c                          |   9 +-
 tests/fdc-test.c               |   4 +-
 tests/qemu-iotests/067.out     | 104 +----
 tests/qemu-iotests/071         |  50 ++-
 tests/qemu-iotests/071.out     |  16 +-
 tests/qemu-iotests/081         |  14 +-
 tests/qemu-iotests/081.out     |   7 +-
 tests/qemu-iotests/087         |   2 +-
 tests/qemu-iotests/087.out     |  16 +-
 tests/qemu-iotests/118         | 638 ++++++++++++++++++++++++++++
 tests/qemu-iotests/118.out     |   5 +
 tests/qemu-iotests/group       |   1 +
 tests/qemu-iotests/iotests.py  |   9 +-
 43 files changed, 2317 insertions(+), 924 deletions(-)
 create mode 100755 tests/qemu-iotests/118
 create mode 100644 tests/qemu-iotests/118.out

-- 
2.4.6