[Qemu-devel] [PATCH v11 00/24] qapi collision reduction (post-introspection subset B')

[Eric Blake]

No pending prerequisites (applies to current master)

Also available as a tag at this location:
git fetch git://repo.or.cz/qemu/ericb.git qapi-cleanupv11b

and I plan to eventually forcefully update my branch with the rest
of the v5 series, at:
http://repo.or.cz/qemu/ericb.git/shortlog/refs/heads/qapi

v11 notes:
Series is definitely converging. Drop patch v10 24/25, and address
comments on several other patches.  Some tests are renamed for more
consistent names, but for the most parts, the changes are localized
and don't have quite the ripple effect that was seen from v9 -> v10.

001/24:[0042] [FC] 'tests/qapi-schema: Test for reserved names, empty struct'
002/24:[----] [--] 'qapi: More idiomatic string operations'
003/24:[0003] [FC] 'qapi: More robust conditions for when labels are needed'
004/24:[0004] [FC] 'qapi: Reserve '*List' type names for list types'
005/24:[0045] [FC] 'qapi: Reserve 'q_*' and 'has_*' member names'
006/24:[----] [--] 'vnc: Hoist allocation of VncBasicInfo to callers'
007/24:[0011] [FC] 'qapi-visit: Split off visit_type_FOO_fields forward decl'
008/24:[0007] [FC] 'qapi-types: Refactor base fields output'
009/24:[0044] [FC] 'qapi: Prefer typesafe upcasts to qapi base classes'
010/24:[0049] [FC] 'qapi: Unbox base members'
011/24:[----] [--] 'qapi-visit: Remove redundant functions for flat union base'
012/24:[0003] [FC] 'qapi: Start converting to new qapi union layout'
013/24:[0004] [FC] 'qapi-visit: Convert to new qapi union layout'
014/24:[----] [-C] 'tests: Convert to new qapi union layout'
015/24:[----] [-C] 'block: Convert to new qapi union layout'
016/24:[----] [--] 'sockets: Convert to new qapi union layout'
017/24:[----] [--] 'net: Convert to new qapi union layout'
018/24:[----] [--] 'char: Convert to new qapi union layout'
019/24:[----] [--] 'input: Convert to new qapi union layout'
020/24:[----] [--] 'memory: Convert to new qapi union layout'
021/24:[----] [--] 'tpm: Convert to new qapi union layout'
022/24:[----] [--] 'qapi: Finish converting to new qapi union layout'
023/24:[0006] [FC] 'qapi: Reserve 'u' member name'
024/24:[0002] [FC] 'qapi: Simplify gen_struct_field()'

v10 notes:
https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg05413.html
Rebase to master, reduce churn in the testsuite (particularly for
things that disappear once the named union conversion is complete),
and rearrange various hunks either forward or backward.  17 patches
turned into 25, because I split things.  Also, several patches got
retitled, which messes up git backport-diff reporting, but some
survived unscathed.

v9 notes:
https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg03730.html
Calling this subset B', because it is mostly new content, but
based heavily on review comments on subset B.  Also, I need to
rebase pending subset C on top of this.  A couple patches
from v5 have been hoisted earlier into the series; while v8
11/18 (Simplify gen_struct_field()) makes an appearance at the
end, and v8 10/18 (Move union tag quirks into subclass) is all
but eliminated.  Since the concepts of these patches has been
on the list for a while, it should be safe to take even during
soft freeze, even if the actual versions of these patches have
lots of new content.

The general goal of this batch of patches is to rework the C
layout of qapi unions so that we get rid of the type/kind
mismatch and isolate tag values from colliding with QMP names.
The tail end of v8 (detecting collisions in check()) will have
to wait a bit longer, but it should be a lot easier to reason
about now that there are fewer collisions possible.

Not worth a backport diff, since most of it is new or previously
unreviewed.

v8 notes:
https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg02879.html
Address review comments, including fixing a bug with tracking
branch name collisions. Include a few more simple test cleanups
(4-6), and add another patch (11) that will make converting from
kind=>type easier in a later subset.  More details in per-patch
changelogs.

v7 notes:
https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg01387.html
Address comments, including a couple of new commits that made
later patches a bit cleaner.  Backport diff gets a bit confused
by a couple of patch titles changing.

v6 notes:
https://lists.gnu.org/archive/html/qemu-devel/2015-10/msg00562.html
This is patches 11-16 of my v5 series; it has grown a bit with
splitting some patches and adding some others.  I suspect that
12/12 on this series will be discarded, but am including it because
it was split from v5 content.

Not much review comments other than on the original 11/46, but there
is enough churn due to rebasing that it's now easier to review this
version than plowing through v5.

Subset C (and more?) will come later.

In v5:
https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg05410.html
I _did_ rearrange patches to try and group related features:

1-2: Groundwork cleanups
3-5: Add more test cases
6-16: Front-end cleanups
17-18: Introspection output cleanups
19-20: 'alternate' type cleanups
21-29: qapi visitor cleanups
30-45: qapi-ify netdev_add
46: add qapi shorthand for flat unions

Lots of fixes based on additional testing, and rebased to
track other changes that happened in the meantime.  The series
is huge; I can split off smaller portions as requested.

In v4:
https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg02580.html
add some more clean up patches
rebase to Markus' recent work
pull in part of Zoltán's work to make netdev_add a flat union,
further enhancing it to be introspectible

I might be able to rearrange some of these patches, or separate
it into smaller independent series, if requested; but I'm
posting now to get review started.

In v3:
https://lists.gnu.org/archive/html/qemu-devel/2015-08/msg02059.html
redo cleanup of dealloc of partial struct
add patches to make all visit_type_*() avoid leaks on failure
add patches to allow boxed command arguments and events

In v2:
https://lists.gnu.org/archive/html/qemu-devel/2015-08/msg00900.html
rebase to Markus' v3 series
rework how comments are emitted for fields inherited from base
additional patches added for deleting colliding 'void *data'
documentation updates to match code changes

v1 was here:
https://lists.gnu.org/archive/html/qemu-devel/2015-07/msg05266.html
https://lists.gnu.org/archive/html/qemu-devel/2015-07/msg05325.html

Eric Blake (24):
  tests/qapi-schema: Test for reserved names, empty struct
  qapi: More idiomatic string operations
  qapi: More robust conditions for when labels are needed
  qapi: Reserve '*List' type names for list types
  qapi: Reserve 'q_*' and 'has_*' member names
  vnc: Hoist allocation of VncBasicInfo to callers
  qapi-visit: Split off visit_type_FOO_fields forward decl
  qapi-types: Refactor base fields output
  qapi: Prefer typesafe upcasts to qapi base classes
  qapi: Unbox base members
  qapi-visit: Remove redundant functions for flat union base
  qapi: Start converting to new qapi union layout
  qapi-visit: Convert to new qapi union layout
  tests: Convert to new qapi union layout
  block: Convert to new qapi union layout
  sockets: Convert to new qapi union layout
  net: Convert to new qapi union layout
  char: Convert to new qapi union layout
  input: Convert to new qapi union layout
  memory: Convert to new qapi union layout
  tpm: Convert to new qapi union layout
  qapi: Finish converting to new qapi union layout
  qapi: Reserve 'u' member name
  qapi: Simplify gen_struct_field()

 block/nbd.c                                        |  18 +--
 block/qcow2.c                                      |  10 +-
 block/vmdk.c                                       |   6 +-
 blockdev.c                                         |  47 +++---
 docs/qapi-code-gen.txt                             |  14 +-
 hmp.c                                              |  26 ++--
 hw/char/escc.c                                     |  12 +-
 hw/input/hid.c                                     |  32 ++---
 hw/input/ps2.c                                     |  24 ++--
 hw/input/virtio-input-hid.c                        |  27 ++--
 hw/mem/pc-dimm.c                                   |   6 +-
 net/dump.c                                         |   4 +-
 net/hub.c                                          |   4 +-
 net/l2tpv3.c                                       |   4 +-
 net/net.c                                          |  24 ++--
 net/slirp.c                                        |   4 +-
 net/socket.c                                       |   4 +-
 net/tap-win32.c                                    |   4 +-
 net/tap.c                                          |   8 +-
 net/vde.c                                          |   4 +-
 net/vhost-user.c                                   |   4 +-
 numa.c                                             |   8 +-
 qemu-char.c                                        | 160 ++++++++++-----------
 qemu-nbd.c                                         |  16 +--
 scripts/qapi-commands.py                           |   4 +-
 scripts/qapi-types.py                              |  68 +++++----
 scripts/qapi-visit.py                              |  90 ++++++------
 scripts/qapi.py                                    |  25 ++--
 spice-qemu-char.c                                  |  12 +-
 tests/Makefile                                     |   8 +-
 tests/qapi-schema/enum-union-clash.err             |   1 -
 tests/qapi-schema/qapi-schema-test.json            |   4 +
 tests/qapi-schema/qapi-schema-test.out             |   3 +
 tests/qapi-schema/reserved-command-q.err           |   1 +
 ...um-union-clash.exit => reserved-command-q.exit} |   0
 tests/qapi-schema/reserved-command-q.json          |   5 +
 ...-base-clash-base.err => reserved-command-q.out} |   0
 tests/qapi-schema/reserved-member-has.err          |   1 +
 tests/qapi-schema/reserved-member-has.exit         |   1 +
 tests/qapi-schema/reserved-member-has.json         |   5 +
 ...num-union-clash.out => reserved-member-has.out} |   0
 tests/qapi-schema/reserved-member-q.err            |   1 +
 tests/qapi-schema/reserved-member-q.exit           |   1 +
 tests/qapi-schema/reserved-member-q.json           |   4 +
 tests/qapi-schema/reserved-member-q.out            |   0
 tests/qapi-schema/reserved-member-u.err            |   1 +
 tests/qapi-schema/reserved-member-u.exit           |   1 +
 tests/qapi-schema/reserved-member-u.json           |   7 +
 tests/qapi-schema/reserved-member-u.out            |   0
 tests/qapi-schema/reserved-type-kind.err           |   1 +
 tests/qapi-schema/reserved-type-kind.exit          |   1 +
 ...um-union-clash.json => reserved-type-kind.json} |   2 -
 tests/qapi-schema/reserved-type-kind.out           |   0
 tests/qapi-schema/reserved-type-list.err           |   1 +
 tests/qapi-schema/reserved-type-list.exit          |   1 +
 tests/qapi-schema/reserved-type-list.json          |   5 +
 tests/qapi-schema/reserved-type-list.out           |   0
 tests/qapi-schema/struct-base-clash-base.exit      |   1 -
 tests/qapi-schema/struct-base-clash-base.json      |   9 --
 tests/qapi-schema/struct-base-clash-base.out       |   5 -
 tests/test-qmp-commands.c                          |  19 +--
 tests/test-qmp-event.c                             |   8 +-
 tests/test-qmp-input-visitor.c                     |  87 +++++------
 tests/test-qmp-output-visitor.c                    |  55 ++++---
 tests/test-visitor-serialization.c                 |  14 +-
 tpm.c                                              |   4 +-
 ui/console.c                                       |  20 +--
 ui/input-keymap.c                                  |  20 +--
 ui/input-legacy.c                                  |  21 +--
 ui/input.c                                         |  84 +++++------
 ui/spice-core.c                                    |  23 +--
 ui/vnc.c                                           | 114 ++++++++-------
 util/qemu-sockets.c                                |  62 ++++----
 73 files changed, 664 insertions(+), 606 deletions(-)
 delete mode 100644 tests/qapi-schema/enum-union-clash.err
 create mode 100644 tests/qapi-schema/reserved-command-q.err
 rename tests/qapi-schema/{enum-union-clash.exit => reserved-command-q.exit} 
(100%)
 create mode 100644 tests/qapi-schema/reserved-command-q.json
 rename tests/qapi-schema/{struct-base-clash-base.err => 
reserved-command-q.out} (100%)
 create mode 100644 tests/qapi-schema/reserved-member-has.err
 create mode 100644 tests/qapi-schema/reserved-member-has.exit
 create mode 100644 tests/qapi-schema/reserved-member-has.json
 rename tests/qapi-schema/{enum-union-clash.out => reserved-member-has.out} 
(100%)
 create mode 100644 tests/qapi-schema/reserved-member-q.err
 create mode 100644 tests/qapi-schema/reserved-member-q.exit
 create mode 100644 tests/qapi-schema/reserved-member-q.json
 create mode 100644 tests/qapi-schema/reserved-member-q.out
 create mode 100644 tests/qapi-schema/reserved-member-u.err
 create mode 100644 tests/qapi-schema/reserved-member-u.exit
 create mode 100644 tests/qapi-schema/reserved-member-u.json
 create mode 100644 tests/qapi-schema/reserved-member-u.out
 create mode 100644 tests/qapi-schema/reserved-type-kind.err
 create mode 100644 tests/qapi-schema/reserved-type-kind.exit
 rename tests/qapi-schema/{enum-union-clash.json => reserved-type-kind.json} 
(69%)
 create mode 100644 tests/qapi-schema/reserved-type-kind.out
 create mode 100644 tests/qapi-schema/reserved-type-list.err
 create mode 100644 tests/qapi-schema/reserved-type-list.exit
 create mode 100644 tests/qapi-schema/reserved-type-list.json
 create mode 100644 tests/qapi-schema/reserved-type-list.out
 delete mode 100644 tests/qapi-schema/struct-base-clash-base.exit
 delete mode 100644 tests/qapi-schema/struct-base-clash-base.json
 delete mode 100644 tests/qapi-schema/struct-base-clash-base.out

-- 
2.4.3