[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Qemu-devel] [PATCH v2 00/29] Modularize generated QAPI code
|
From: |
Markus Armbruster |
|
Subject: |
[Qemu-devel] [PATCH v2 00/29] Modularize generated QAPI code |
|
Date: |
Sun, 11 Feb 2018 10:35:38 +0100 |
Our qapi-schema.json is composed of modules connected by include
directives, but the generated code is monolithic all the same: one
qapi-types.h with all the types, one qapi-visit.h with all the
visitors, and so forth. These monolithic headers get included all
over the place. In my "build everything" tree, adding a QAPI type
recompiles about 4800 out of 5100 objects.
We wouldn't write such monolithic headers by hand. It stands to
reason that we shouldn't generate them, either.
This series' basic idea is to split up generated headers to mirror the
schema's modular structure: one header per module. That way, you can
include just what you need.
The compile-time improvements can be massive. Before this series, any
QAPI schema change recompiles some 4800 out of 5100 objects in my
"build everything" tree. Afterwards, adding a type to
qapi/migration.json recompiles a bit over 100, and a documentation
change no longer recompiles anything.
Related: Marc-André's 'unit' pragma proposal. That's a different way
to split off parts of the generated code, motivated by the desire to
use poisoned identifiers such as TARGET_I386. I noted in my review of
v3 that I "can either accept it, or come up with a better solution."
This is my attempt at a better solution. It's a bit more ambitious,
and thus more useful (I hope). The pragma has one theoretical
advantage, though: you can modularize the generated output in
different ways than the input. The patches using don't do that,
however.
v2:
* Rebased, annoying conflicts due to the Python 2 rigmarole
* PATCH 01: New
* PATCH 02-03,08,10-11,17,19-24: Commit messages improved
[Eric, Marc-André]
* PATCH 04: Split off the next patch [Eric]
* PATCH 05,07-08,10,19: QAPIGen methods renamed [Marc-André]
* PATCH 08,20,22: fix missing .gitignore updates
* PATCH 08:
- generated doc renamed to qapi-doc.texi
- qapi-code-gen.txt updated
- useless options -c, -h dropped
* PATCH 09: New, replacing old PATCH 07
* PATCH 16: Latent bug in QAPISchema.check() fixed
* PATCH 18: New
* PATCH 19-20,22-24:
- Common code factored out into QAPISchemaMonolithicCVisitor
and QAPISchemaModularCVisitor
- All code-generating visitors covered
- Consistent parameter order prefix, opt_builtins
- R-bys dropped
* PATCH 25-29: New
Markus Armbruster (29):
Include qapi/qmp/qerror.h exactly where needed
qapi: Streamline boilerplate comment generation
qapi: Generate up-to-date copyright notice
qapi: Rename variable holding the QAPISchemaGenFOOVisitor
qapi: New classes QAPIGenC, QAPIGenH, QAPIGenDoc
qapi: Reduce use of global variables in generators some
qapi: Turn generators into modules
qapi-gen: New common driver for code and doc generators
qapi-gen: Convert from getopt to argparse
qapi: Touch generated files only when they change
qapi: Improve include file name reporting in error messages
qapi/common: Eliminate QAPISchema.exprs
qapi: Lift error reporting from QAPISchema.__init__() to callers
qapi: Concentrate QAPISchemaParser.exprs updates in .__init__()
qapi: Record 'include' directives in parse tree
qapi: Generate in source order
qapi: Record 'include' directives in intermediate representation
qapi: Rename generated qmp-marshal.c to qmp-commands.c
qapi: Make code-generating visitors use QAPIGen more
qapi/types qapi/visit: Generate built-in stuff into separate files
qapi/common: Fix guardname() for funny filenames
qapi: Generate separate .h, .c for each module
Include less of the generated modular QAPI headers
qapi: Empty out qapi-schema.json
docs/devel/writing-qmp-commands: Update for modular QAPI
docs: Correct outdated information on QAPI
qapi: Move qapi-schema.json to qapi/, rename generated files
Fix up dangling references to qmp-commands.* in comment and doc
qapi: Don't create useless directory qapi-generated
.gitignore | 80 ++++-
Makefile | 233 ++++++++++----
Makefile.objs | 80 ++++-
arch_init.c | 2 +-
backends/cryptodev.c | 1 -
backends/hostmem.c | 3 +-
balloon.c | 2 +-
block.c | 2 -
block/block-backend.c | 2 +-
block/crypto.c | 2 +-
block/iscsi.c | 2 +-
block/nbd.c | 2 +-
block/nfs.c | 2 +-
block/qapi.c | 4 +-
block/qcow2.c | 4 +-
block/quorum.c | 2 +-
block/sheepdog.c | 2 +-
block/ssh.c | 2 +-
block/throttle-groups.c | 2 +-
block/write-threshold.c | 4 +-
blockdev-nbd.c | 2 +-
blockdev.c | 5 +-
blockjob.c | 2 +-
chardev/char-fe.c | 2 +-
chardev/char-ringbuf.c | 2 +-
chardev/char-socket.c | 1 +
chardev/char.c | 4 +-
configure | 1 -
cpus.c | 4 +-
crypto/cipherpriv.h | 2 +-
docs/devel/qapi-code-gen.txt | 119 ++++---
docs/devel/writing-qmp-commands.txt | 39 +--
docs/interop/qmp-intro.txt | 3 +-
docs/xen-save-devices-state.txt | 3 +-
dump.c | 4 +-
hmp.c | 12 +-
hw/acpi/core.c | 4 +-
hw/acpi/cpu.c | 2 +-
hw/acpi/memory_hotplug.c | 2 +-
hw/acpi/vmgenid.c | 2 +-
hw/block/block.c | 1 +
hw/block/hd-geometry.c | 1 +
hw/char/virtio-console.c | 2 +-
hw/core/machine.c | 2 +-
hw/core/qdev.c | 2 +-
hw/i386/pc.c | 2 +-
hw/i386/xen/xen-hvm.c | 2 +-
hw/ipmi/ipmi.c | 2 +-
hw/mem/nvdimm.c | 1 -
hw/net/rocker/qmp-norocker.c | 2 +-
hw/net/rocker/rocker.c | 2 +-
hw/net/rocker/rocker_fp.c | 2 +-
hw/net/rocker/rocker_of_dpa.c | 2 +-
hw/net/virtio-net.c | 2 +-
hw/pci/pci-stub.c | 2 +-
hw/pci/pci.c | 2 +-
hw/ppc/spapr_rtas.c | 1 -
hw/ppc/spapr_rtc.c | 2 +-
hw/s390x/s390-skeys.c | 2 +-
hw/timer/mc146818rtc.c | 4 +-
hw/tpm/tpm_emulator.c | 1 +
hw/tpm/tpm_passthrough.c | 1 +
hw/virtio/virtio-balloon.c | 2 +-
hw/watchdog/watchdog.c | 4 +-
include/block/block.h | 2 +-
include/block/dirty-bitmap.h | 2 +-
include/block/nbd.h | 2 +-
include/chardev/char.h | 1 +
include/crypto/cipher.h | 2 +-
include/crypto/hash.h | 2 +-
include/crypto/hmac.h | 2 +-
include/crypto/secret.h | 1 +
include/crypto/tlscreds.h | 1 +
include/hw/block/block.h | 2 +-
include/hw/block/fdc.h | 2 +-
include/hw/ppc/spapr_drc.h | 1 +
include/hw/qdev-properties.h | 2 +
include/io/dns-resolver.h | 1 +
include/migration/colo.h | 2 +-
include/migration/failover.h | 2 +-
include/migration/global_state.h | 1 +
include/monitor/monitor.h | 1 +
include/net/filter.h | 1 +
include/net/net.h | 2 +-
include/qapi/clone-visitor.h | 1 -
include/qapi/error.h | 2 +-
include/qapi/qmp/qobject.h | 2 +-
include/qapi/visitor.h | 2 +-
include/qemu/sockets.h | 2 +-
include/qemu/throttle.h | 2 +-
include/qom/cpu.h | 1 +
include/qom/object.h | 2 +-
include/sysemu/arch_init.h | 2 +-
include/sysemu/balloon.h | 2 +-
include/sysemu/dump.h | 2 +
include/sysemu/hostmem.h | 1 +
include/sysemu/replay.h | 2 +
include/sysemu/sysemu.h | 1 +
include/sysemu/tpm.h | 1 +
include/sysemu/watchdog.h | 2 +-
include/ui/input.h | 2 +-
io/channel-socket.c | 1 +
io/dns-resolver.c | 1 +
iothread.c | 2 +-
migration/colo-failover.c | 2 +-
migration/colo.c | 2 +-
migration/migration.c | 4 +-
migration/migration.h | 1 +
migration/ram.c | 2 +-
migration/ram.h | 2 +-
migration/savevm.c | 3 +-
monitor.c | 8 +-
net/colo-compare.c | 1 -
net/filter-buffer.c | 2 +-
net/filter-mirror.c | 1 -
net/filter-rewriter.c | 1 -
net/net.c | 4 +-
net/tap_int.h | 2 +-
net/vhost-user.c | 2 +-
numa.c | 4 +-
qapi-schema.json => qapi/misc.json | 105 +-----
qapi/qapi-schema.json | 95 ++++++
qapi/run-state.json | 10 +
qdev-monitor.c | 2 +-
qemu-img.c | 2 +-
qga/Makefile.objs | 2 +-
qga/commands-posix.c | 2 +-
qga/commands-win32.c | 2 +-
qga/commands.c | 2 +-
qga/main.c | 2 +-
qmp.c | 18 +-
qom/object.c | 2 +-
qom/object_interfaces.c | 2 +-
replay/replay-input.c | 1 +
replication.h | 1 +
scripts/qapi-gen.py | 57 ++++
scripts/qapi/__init__.py | 0
scripts/{qapi-commands.py => qapi/commands.py} | 155 ++++-----
scripts/{qapi.py => qapi/common.py} | 355 +++++++++++++--------
scripts/{qapi2texi.py => qapi/doc.py} | 92 +++---
scripts/{qapi-event.py => qapi/events.py} | 128 +++-----
scripts/{qapi-introspect.py => qapi/introspect.py} | 123 +++----
scripts/{qapi-types.py => qapi/types.py} | 185 ++++-------
scripts/{qapi-visit.py => qapi/visit.py} | 189 ++++-------
stubs/tpm.c | 3 +-
stubs/uuid.c | 2 +-
stubs/vmgenid.c | 2 +-
stubs/xen-hvm.c | 2 +-
target/arm/monitor.c | 3 +-
target/i386/cpu.c | 4 +-
target/s390x/kvm.c | 1 -
tests/.gitignore | 9 +-
tests/Makefile.include | 70 ++--
tests/qapi-schema/comments.out | 3 +-
tests/qapi-schema/doc-bad-section.out | 5 +-
tests/qapi-schema/doc-good.out | 33 +-
tests/qapi-schema/doc-good.texi | 3 +-
tests/qapi-schema/empty.out | 2 +-
tests/qapi-schema/event-case.out | 3 +-
tests/qapi-schema/ident-with-escape.out | 7 +-
tests/qapi-schema/include-no-file.err | 2 +-
tests/qapi-schema/include-relpath.out | 7 +-
tests/qapi-schema/include-repetition.out | 12 +-
tests/qapi-schema/include-simple.out | 5 +-
tests/qapi-schema/indented-expr.out | 3 +-
tests/qapi-schema/qapi-schema-test.out | 321 +++++++++----------
tests/qapi-schema/test-qapi.py | 19 +-
tests/qmp-test.c | 3 +-
tests/test-char.c | 2 +-
tests/{test-qmp-commands.c => test-qmp-cmds.c} | 2 +-
tests/test-qmp-event.c | 3 +-
tests/test-qobject-input-visitor.c | 8 +-
tests/test-visitor-serialization.c | 1 -
tpm.c | 3 +-
trace/qmp.c | 2 +-
ui/cocoa.m | 2 +-
ui/console.c | 2 +-
ui/gtk.c | 2 +-
ui/input-legacy.c | 2 +-
ui/input.c | 2 +-
ui/spice-core.c | 4 +-
ui/vnc.c | 4 +-
ui/vnc.h | 1 +
util/qemu-config.c | 2 +-
util/qemu-sockets.c | 2 +-
vl.c | 8 +-
186 files changed, 1556 insertions(+), 1352 deletions(-)
rename qapi-schema.json => qapi/misc.json (95%)
create mode 100644 qapi/qapi-schema.json
create mode 100755 scripts/qapi-gen.py
create mode 100644 scripts/qapi/__init__.py
rename scripts/{qapi-commands.py => qapi/commands.py} (67%)
rename scripts/{qapi.py => qapi/common.py} (90%)
rename scripts/{qapi2texi.py => qapi/doc.py} (78%)
mode change 100755 => 100644
rename scripts/{qapi-event.py => qapi/events.py} (61%)
rename scripts/{qapi-introspect.py => qapi/introspect.py} (71%)
rename scripts/{qapi-types.py => qapi/types.py} (53%)
rename scripts/{qapi-visit.py => qapi/visit.py} (64%)
rename tests/{test-qmp-commands.c => test-qmp-cmds.c} (99%)
--
2.13.6
- [Qemu-devel] [PATCH v2 00/29] Modularize generated QAPI code,
Markus Armbruster <=