[Qemu-devel] [RFC] New Migration Protocol using Visitor Interface

[Michael Roth]

OVERVIEW

This patch series implements a QEMUFile Visitor class that's intended to 
abstract away direct calls to qemu_put_*/qemu_get_* for save/load functions. 
Currently this is done by always creating a 
QEMUFileInputVisitor/QEMUFileOutputVisitor pair with each call to 
qemu_fopen_ops() and maintaining a QEMUFile->I/O Visitor mapping. save/load 
functions that are to be converted would them simply use lookup routines to get 
a Visitor based on their QEMUFile arguments. Once these save/load functions are 
all converted, we can then change the interfaces in bulk and switch over to 
passing in the Visitor directly.

An example conversion of Slirp, which still uses old-style save/load routines, 
is included as part of this series. Anthony I believe has VMState converted 
over to using Visitors, so theoretically all VMStatified devices are good to go 
(Anthony: if that's the case feel free to rebase on this or point me to the 
repo to pull in, and I'll work off the resulting branch).

PLANS

Ultimately, the goal is to implement a new migration protocol that is 
self-describing, such that incompatibilities or migration errors can be more 
easilly detected. Currently, a simple change in data types for a particular 
device can introduce subtle bugs that won't be detected by the target, since 
the target interprets the data according to it's own expectation of what those 
data types are. Currently the plan is to use ASN.1 BER in place of QEMUFile.

By using a Visitor interface for migration, we also gain the ability to 
generate a migration schema (via, say, a JSONSchemaVisitor). This is similar to 
the VMState schema generator in Anthony's "Add live migration unit tests" 
series (http://thread.gmane.org/gmane.comp.emulators.qemu/97754/focus=97754), 
except that it is applicable to both VMState and old-style save/load functions.

There is still quite a bit a work to get to that point. Anthony addressed some 
of the issues on the VMState side of things in the aforementioned series, and I 
believe he already has some patches that convert VMState over to using visitors 
insteads of qemu_put_*/qemu_get_* directly. That being the case, what's left is:

1) Convert old-style save/load functions over to using visitors. I'm not sure 
what the best approach is for this. We basically have 2 options: either by 
converting them to VMState, or converting the functions over to using visitors, 
such as with the example slirp conversion. Even if we do option 2 it would 
likely need to be done in 2 phases:

phase 1: a mechanical conversion where we basically replace each 
qemu_put*/qemu_get* with a visit_*. This allows use to plop in a new, say 
BERVisitor to switch to using the new migration protocol. It should also be 
possible to do this without breaking migration compatibility with older 
versions. It does not however result in a well-specified schema if we were to 
plop in, say, a JSONSchemaVisitor. This is where we need phase 2.

phase 2: Currently, runtime state modifies our "schema" the way things are 
currently done. Slirp instance and virtqueue enumeration, for instance:

slirp/slirp.c:
    for (ex_ptr = slirp->exec_list; ex_ptr; ex_ptr = ex_ptr->ex_next)
        if (ex_ptr->ex_pty == 3) {
            struct socket *so;
            so = slirp_find_ctl_socket(slirp, ex_ptr->ex_addr,
                                       ntohs(ex_ptr->ex_fport));
            if (!so)
                continue;

            qemu_put_byte(f, 42);
            slirp_socket_save(f, so);
        }

hw/virtio.c:
    for (i = 0; i < VIRTIO_PCI_QUEUE_MAX; i++) {
        if (vdev->vq[i].vring.num == 0)
            break;

        qemu_put_be32(f, vdev->vq[i].vring.num);
        qemu_put_be64(f, vdev->vq[i].pa);
        qemu_put_be16s(f, &vdev->vq[i].last_avail_idx);
        if (vdev->binding->save_queue)
            vdev->binding->save_queue(vdev->binding_opaque, i, f);
    }


To be able to build a schema around this we'd need to do a visit_start_array() 
or visit_start_list(), to denote a list of entries, and ideally we'd also 
describe structure of these entries. But often there is no structure of these 
entries...they may just be values taken from various data structures. So, 
either we do hacky things like using visitor intefaces to create "fake" structs 
(we say we're dealing with a struct when we're really not), or we actually put 
these into an intermediate struct which we then use to assign to store/load 
migration fields from.

phase 2 sounds a whole lot like converting over to VMState. So I think we'll 
only want to go with option 2 in places where converting to VMState is not 
practical/planned. Or we can hold off on the phase 2 stuff and just focus on 
getting the new protocol in place ASAP. After which, we can rework things to 
support generating a well-specified schema.

2) Once the conversion stuff is done, we can modify the save/load interfaces in 
bulk to accept a Visitor in place of a QEMUFile (but continue using the old 
protocol/QEMUFile*Visitor for the time being)

3) Implement migration capabilities negotiation. This was discussed to some 
extent in the XBZRLE v4 thread 
(http://lists.gnu.org/archive/html/qemu-devel/2011-08/msg01155.html), basically 
having a set of migration capabilities that can be queried via QMP. Management 
tools would then choose the optimal migration settings from the intersection of 
these.

4) Implement the BERVisitor and make this the default migration protocol.

Most of the work will be in 1), though with the implementation in this series 
we should be able to do it incrementally. I'm not sure if the best approach is 
doing the mechanical phase 1 conversion, then doing phase 2 sometime after 4), 
doing phase 1 + 2 as part of 1), or just doing VMState conversions which gives 
basically the same capabilities as phase 1 + 2.

Thoughts?

 Makefile                        |    4 +-
 Makefile.objs                   |   19 +-
 cutils.c                        |    8 +
 hw/hw.h                         |   20 ++
 qapi/qapi-visit-core.c          |   78 ++++++
 qapi/qapi-visit-core.h          |   30 ++
 qapi/qemu-file-input-visitor.c  |  350 +++++++++++++++++++++++
 qapi/qemu-file-input-visitor.h  |   26 ++
 qapi/qemu-file-output-visitor.c |  353 +++++++++++++++++++++++
 qapi/qemu-file-output-visitor.h |   26 ++
 qemu-common.h                   |    1 +
 qemu-file.c                     |  585 +++++++++++++++++++++++++++++++++++++++
 savevm.c                        |  494 ---------------------------------
 slirp/slirp.c                   |  366 ++++++++++++++----------
 test-visitor.c                  |  404 +++++++++++++++++++++++++++
 15 files changed, 2110 insertions(+), 654 deletions(-)