Re: [Qemu-devel] [RFC PATCH 0/4] Fix subsection ambiguity in the migration format

[Juan Quintela]

Anthony Liguori <address@hidden> wrote:
> On 07/25/2011 04:10 PM, Paolo Bonzini wrote:

> == Today ==
>
> Today we only support generating the latest serialization of
> devices. To increase the probability of the latest version working on
> older versions of QEMU, we strategically omit fields that we know can
> safely be omitted with older versions (subsections).  More than
> likely, migrating new to old won't work.
>
> Migrating old to new is more likely to work.  We version each section
> in order to be able to identify when we're dealing with old.
>
> But all of this logic lives in one of two forms.  Either as a
> savevm/loadvm callback that takes a QEMUFile and writes byte
> serialization to the stream in an open way (usually big endian) or
> encoded declaratively in a VMState section.

We have a very "poor" way to try to load a device without some features,
but support is very bad.

> == What we need ==
>
> We need to decompose migration into three different problems: 1)
> serializing device state 2) transforming the device model in order to
> satisfy forwards and backwards compatibility 3) encoding the
> serialized device model on the wire.

I will change this to:
- We need to be able to "enable/disable" features of a device.
  A.K.A. make -M pc-0.14 work with devices with the same features
  than 0.14.  Notice that this is _independent_ of migration.

- Be able to describe that different features/versions.  This is not the
  difficult part, it can be subsections, optional fields, whatever.
  What is the difficult part is _knowing_ what fields needs to be on
  each version.  That again depends of the device, not migration.

- Be able to to do forward/bacward compatibility (and without
  comunication both sides is basically impossible).

- Send things on the wire (really this is the easy part, we can play
  with it touching only migration functions.).

> We also need a way to future proof ourselves.

We have been very bad at this.  Automatic checking is the only way that
I can think of.

> == What we can do ==
>
> 1) Add migration capabilities to future proof ourselves.  I think the
> simplest way this would work is to have a
> query-migration-capabilities' command that returned a bitmask of
> supported migration features.  I think we also introduce a
> set-migration-capabilities' command that can mask some of the
> supported features.

We have two things here.  Device level & protocol level.

Device level: very late to set anything.
Protocol level: we can set things here, but notice that only a few
things cane be set here.

> A management tool would query-migration features on the source and
> destination, take the intersection of the two masks, and set that mask
> on both the source and destination.
>
> Lack of support for these commands indicates a mask of zero which is
> the protocol we offer today.
>
> 2) Switch to a visitor model to serialize device state.  This involves
> converting any occurance of:
>
> qemu_put_be32(f, port->guest_connected);
>
> To:
>
> visit_type_u32(v, "guest_connected", &port->guest_connected, &local_err);

VMSTATE_INT32(guest_conected, FooState)

can be make to do this at any point.

> It's 100% mechanical and makes absolutely no logic change.  It works
> equally well with legacy and VMstate migration handlers.
>
> 3) Add a Visitor class that operates on QEMUFile.
>
> At this state, we can migrate to data structures.  That means we can
> migrate to QEMUFile, QObjects, or JSON.  We could change the protocol
> at this stage to something that was still binary but had section sizes
> and things of that nature.

That was the whole point of vmstate.

> But we shouldn't stop here.
>
> 4) Compatibility logic should be extracted from the savevm functions
> and VMstate functions into separate functions that take a data
> structure. Basically, we want to have something roughly equivalent to:
>
> QObject *e1000_migration_compatibility(QObject *src, int src_version,
> int dst_version);
>
> We can have lots of helpers that reuse the VMstate declarative stuff
> to do this but this should be registered independent of the main
> serialization handler.
>
> This moves us to a model where we always generate the latest
> serialization format, and then have specific ways to convert to older
> mechanisms.  It allows us to do very big backwards compatibility steps
> like convert the state of one device into two separate devices
> (because we're just dealing with in-memory data structures).

Paint me sceptic about this.   I don't think this is going to work
because that functions will rote very fast.

> It's this step that lets us truly support compatibility with
> migration. The good news is, it doesn't have to be all or nothing.
> Since we always already generate the latest serialization format, the
> existing code only deals with migrating older versions to the latest
> which is something that isn't all that important.
>
> So if we did this in 1.0, we could have a single function that
> converted the 1.0 device model to 1.1 and vice versa, and we'd be
> fine.  We wouldn't have to touch 200 devices to do this.

I still think this is wrong.  We are launching a device with feature
"foo", and at migration time, we want to migration without feature
"foo".  This is not going to work on the general case.  But launching
the device _without_ feature "foo" will always work.

Notice the things that "can" be optional:
- features that are not used.  We update the device to have more
  features, but OS driver only uses the features of the old version.
  With subsections test, we can fix this one.

- values that are only needed sometimes.  PIO subsection cames to mind,
  it is only needed when we are on the middle of a PIO operation.

- values that rarely change for defaults.  This the mmio addresess
  problems with rtl8139.  If we plug/unplug the card, we will get a
  different address, so we need to change it.

- values that depend of other features (change default size of memory,
  add new variables, etc).  This is for its very nature not compatible,
  and we can't migrate.

What I am complaining here?  This "compatibility" support supposes that
migration works as:

device with some features -> migration -> device with other features

and it works.  This means that "migration" does magic, and this is never
going to work.

Until now, this kind of worked because we only supported migration from
old -> new, or the same version.  Migration from old -> new can never
have new features.  But from new -> old to work, we need a way to
disable the new features.   That is completely independent of migration.


> 5) Once we're here, we can implement the next 5-year format.  That
> could be ASN.1 and be bidirectional or whatever makes the most sense.
> We could support 50 formats if we wanted to.  As long as the transport
> is distinct from the serialization and compat routines, it really
> doesn't matter.

This means finishing the VMState support, once there, only thing needs
to change is "copy" the savevm, and change the "visitors" to whatever
else that we need/want.

Later, Juan.