[Qemu-devel] [PATCH v4 00/12] qapi: Allow blockdev-add for NBD

[Max Reitz]

This series adds blockdev-add support for NBD clients.

Good news in v4: The total diffstat changed from 443+/98- to 407+/106-.
Bad news in v4:  10/12 patches have functional differences from v3.


Patches 1, 2, 3, and 4 are minor patches with no functional relation to
this series, other than the fact that later patches will touch the code
they touch, too.

Patch 5 prepares the code for the addition of a new option prefix, which
is "address.".

Patch 6 makes the NBD client accept a SocketAddress under the "address"
option (or rather, a flattened SocketAddress QDict with its keys
prefixed by "address."). The old options "host", "port", and "path" are
supported as legacy options and translated to the respective
SocketAddress representation.

Patch 7 drops usage of "host", "port", and "path" outside of
nbd_has_filename_options_conflict(),
nbd_process_legacy_socket_options(), and nbd_refresh_filename(), making
those options nothing but legacy.

Patch 8, the goal of this series, is again not very complicated.

Patches 9, 10, and 11 are required for the iotest added in patch 12. It
will invoke qemu-nbd, so patch 9 is required. Besides qemu-nbd, it will
launch an NBD server VM concurrently to the client VM, which is why
patch 10 is required. And finally, it will test whether we can add an
NBD BDS by passing it a file descriptor, which patch 11 is needed for
(so we use the socket_scm_helper to pass sockets to qemu).

Patch 12 then adds the iotest for NBD's blockdev-add interface.


*** This series depends on the following patches: ***
- "qdict: implement a qdict_crumple method for un-flattening a dict"
  from Dan's "QAPI/QOM work for non-scalar object properties" series
- "qemu-nbd: Add --fork option" from my "iotests: Fix test 162" series


v2:
- Dropped patch 1; NBD now uses QemuOpts (as it is supposed to) for the
  legacy options, therefore we cannot just rename keys in the options
  QDict, so the function introduced in the old patch 1
  (qdict_change_key()) becomes obsolete.
- Patch 2 (prev. 3): Rebase conflicts due to the QemuOpts change
- Patch 3 (prev. 4): QemuOpts rebase conflicts
- Patch 4 (prev. 5): QemuOpts rebase conflicts
- Patch 5 (prev. 6):
  - Spell fix in the commit message
  - Break if () condition more consistent with the rest of qemu [Eric]
- Patch 6:
  - Merged previous patches 7 and 8 into a single one; this is because
    of the next point:
  - Use the QMP output visitor to generate the BDS options; this makes
    the resulting code shorter and working even for other socket types
    then 'unix' and 'inet' (i.e. 'fd');
    for this to work, however, we have to merge said two patches because
    the filename refreshing code (previously patch 7) now needs the
    SocketAddress object parsed by the code previously introduced by
    patch 8
    (and the refreshing code needs to be there before the parsing code,
    so we cannot invert the order either)
  - Rebase conflicts because of the changes in patch 5
  - The QemuOpts change means we can no longer just rename the legacy
    options in the options QDict; instead, we have to fetch them using
    the QemuOpts functions and then build a new options QDict from
    scratch.
- Patch 7 (prev. 9):
  - Dropped some changes that become unnecessary now that we use the QMP
    output visitor for nbd_refresh_filename()
- Patch 8 (prev. 10):
  - Simple rebase conflict in the QAPI schema block driver list
  - Changed "Since: 2.6" to "Since: 2.8"
- Patch 9 (prev. 11):
  - Some contextual rebase conflicts
  - iotests.py no longer sets __all__
  - Use --fork instead of a pipe to run it in the background (so we
    don't have to wait an arbitrary timespan after calling the function
    until the server is actually started)
- Patch 10 (prev. 12): Rebase conflicts due to the refactoring of the VM
  class structure
- Patch 12 (prev. 14):
  - Fixed descriptive comment (what the test does)
  - blockdev-add no longer takes an @id parameter, so use @node-name
    instead (and change the rest so it works)
  - Pack os.remove() of socket files into a try-except block


git-backport-diff against v3:

Key:
[----] : patches are identical
[####] : number of functional differences between upstream/downstream patch
[down] : patch is downstream-only
The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively

001/12:[----] [-C] 'block/nbd: Drop trailing "." in error messages'
002/12:[0005] [FC] 'block/nbd: Reject port parameter without host'
003/12:[0031] [FC] 'block/nbd: Default port in nbd_refresh_filename()'
004/12:[0017] [FC] 'block/nbd: Use qdict_put()'
005/12:[0008] [FC] 'block/nbd: Add nbd_has_filename_options_conflict()'
006/12:[0193] [FC] 'block/nbd: Accept SocketAddress'
007/12:[0008] [FC] 'block/nbd: Use SocketAddress options'
008/12:[0010] [FC] 'qapi: Allow blockdev-add for NBD'
009/12:[0007] [FC] 'iotests.py: Add qemu_nbd function'
010/12:[0013] [FC] 'iotests.py: Allow concurrent qemu instances'
011/12:[----] [--] 'socket_scm_helper: Accept fd directly'
012/12:[0035] [FC] 'iotests: Add test for NBD's blockdev-add interface'


Max Reitz (12):
  block/nbd: Drop trailing "." in error messages
  block/nbd: Reject port parameter without host
  block/nbd: Default port in nbd_refresh_filename()
  block/nbd: Use qdict_put()
  block/nbd: Add nbd_has_filename_options_conflict()
  block/nbd: Accept SocketAddress
  block/nbd: Use SocketAddress options
  qapi: Allow blockdev-add for NBD
  iotests.py: Add qemu_nbd function
  iotests.py: Allow concurrent qemu instances
  socket_scm_helper: Accept fd directly
  iotests: Add test for NBD's blockdev-add interface

 block/nbd.c                            | 225 ++++++++++++++++++++-------------
 qapi/block-core.json                   |  25 +++-
 tests/qemu-iotests/051.out             |   4 +-
 tests/qemu-iotests/051.pc.out          |   4 +-
 tests/qemu-iotests/147                 | 201 +++++++++++++++++++++++++++++
 tests/qemu-iotests/147.out             |   5 +
 tests/qemu-iotests/group               |   1 +
 tests/qemu-iotests/iotests.py          |  14 +-
 tests/qemu-iotests/socket_scm_helper.c |  29 +++--
 9 files changed, 402 insertions(+), 106 deletions(-)
 create mode 100755 tests/qemu-iotests/147
 create mode 100644 tests/qemu-iotests/147.out

-- 
2.10.0