[Qemu-devel] [RFC] Use QEMU IDL for device serialization/vmstate

[Michael Roth]

These patches are based on qom-next, and can also be obtained from:

git://github.com/mdroth/qemu.git qidl-rfc1

= Overview =

The goal of these patches is to explore how we can leverage an IDL to improve
device serialization/migration.

Patches 1-7 introduce the QIDL compiler and QAPI modification to handle the
QAPI schemas it generates.

Patches 8-12 adds build infrastructure and QIDL annotations to RTC which are
then use to generate visitors which expose RTC's device state via a new QOM
property.

Patches 13-17 adds infrastructure for generating vmstate field descriptions
for RTC using those same annotations.

The procedure for subsequent devices is then, in ideal cases at least, mostly
a matter of moving the device state definitions to header files similar
fashion, adding QIDL annotations, and adding header files to the lists
QIDL_SOURCE and QIDL_SOURCE_VMSTATE lists in Makefile.target.

Non qom-ified devices, or non-devices, can still be handled with open-coded
visitors and legacy registration facilities.

= General/Future Plans =

This is all very much open to discussion, and I only have a general idea of
how we can leverage this to improve migration compatibility/support. That
said:

With everything in place, we'd now have a means to serialize device state into
QObjects (or whatever). We can then perform whatever transformations/mark-up
we need to do (based on capabilities negotation centering around per-device
capabilities, for example), and send the state over the wire for migration.

The wire protocol is then simply a matter of how we encode the transformed
QObject. So a BER protocol could be implemented by creating something analagous
to the JSON-encoding facilities in qjson.c. Or, we could just keep using JSON,
perhaps with compression on top.

Eventually we can extend this approach to send device properties and encode the
actual composition tree in such a way that we can create machine machines on
the target side and avoid the need to duplicate the command-line invocation,
though that will take some substantial re-architecting/removal of the various
factory interfaces and further QOMification.

I'm not planning on incorporating memory migration into this, but it may be
possible to extend this approach to hotplug events/data blocks as well.

Thoughts?

= TODOs =

 - complete QIDL-fication for all common x86 devices
 - make the build system more robust
 - `make check` tests to exercise the automatic detection of device state
   changes (though dirty git trees may be sufficient in-and-of themselves)
 - Work in prototype or considerations for a migration protocol to leverage
   this

 Makefile.objs                        |    6 +
 Makefile.target                      |   52 +++-
 hw/mc146818rtc.c                     |   60 +---
 hw/mc146818rtc_state.h               |   33 ++
 qapi/misc-qapi-visit.c               |   35 ++
 qapi/qapi-visit-core.c               |   25 ++
 qapi/qapi-visit-core.h               |    8 +
 qapi/qc.h                            |   18 +
 qapi/qmp-output-visitor.c            |   21 ++
 qidl-generated/mc146818rtc.json      |    1 +
 qidl-generated/mc146818rtc_vmstate.h |   20 ++
 scripts/qapi-commands.py             |    8 +-
 scripts/qapi-types.py                |    2 +-
 scripts/qapi-visit.py                |   90 +++++-
 scripts/qapi.py                      |   10 +-
 scripts/qc.md                        |  331 +++++++++++++++++++
 scripts/qc.py                        |  599 ++++++++++++++++++++++++++++++++++
 17 files changed, 1255 insertions(+), 64 deletions(-)