Re: [Qemu-devel] [RFC] New Migration Protocol using Visitor Interface

[Anthony Liguori]

On 10/03/2011 09:15 AM, Michael S. Tsirkin wrote:
> On Mon, Oct 03, 2011 at 08:43:54AM -0500, Anthony Liguori wrote:
> > Having the ability to ignore some fields is not enough.
>
> But it is also really required.

I agree.  It's the principle of being conservative in what you send and liberal 
in what you accept.

> > We need to
> > also be able to split a single field into multiple fields, and event
> > split a single device into multiple devices.  If we're dealing with
> > a tree data structure in memory, we have limitless ability to fudge
> > things.
> >
> > Visitors buy us this.  Right now, we're talking about:
> >
> > void virtio_marshal(QEMUFile *f, VirtioDevice *vdev);
> >
> > If you just do BER encoding, that doesn't help us.  We still can
> > only read from the wire to a VirtioDevice.   If we do:
> >
> > void virtio_marshal(Visitor *v, VirtioDevice *vdev);
> >
> > We can still do BER encoding.  We can also do QEMUFile encoding in
> > the current format.
> >
> > But we also can Visit from a QObject.  That means we can write
> > something that reads in BER off the wire and generates QObjects
> > corresponding to whatever is sent.  We can then manipulate those
> > QObjects in memory as much as we'd like before handing a QObject
> > Visitor to virtio_marshal.
> >
> > Separating parsing off the wire from handing something to the device
> > models and allowing transformation in between is the key to
> > addressing compatibility in a robust fashion.
>
> Yes. I agree it is required, and doing this with visitors
> does looks like a good way to handle that.
>
> However the ability to skip fields at the protocol level
> is also required, and it needs to be handled at the
> protocol level. For example, I am worried about
> invalid input with a huge array size making us allocate
> and read a gigabyte into memory on the assumption that we will want to
> manipulate it later. Yes we could introduce artificial
> buffer size limits but these might then come to bite us.
> If instead we can see that no one needs the array
> up front, it will be easy to discard.

Yes, we just were covering in the other parts of the thread.  When you 
end_struct, end_array, or end_list, the visitor can discard whatevers left.

How I see this all evolving in the future is that we would have a formal 
protocol specification.  From that spec, we would generate Visitors.  This would 
handle taking what's on the wire and building an in-memory tree.  If an ASN.1 
sequence that isn't part of the spec is passed, we should just silently discard it.

I think the scenario above is that we have an array in the spec, but the 
implementation no longer uses that array.  Your concern is that its inefficient 
to keep it in memory?

I think there are two cases.  The first is that that array is required in the 
spec, but optional in our implementation and that decision is made at run time. 
 In this case, it needs to get entered into memory.

If the array is no longer needed at all in our implementation, we can certainly 
annotate the protocol spec to skip it and not bring it into memory.  Maybe it's 
an internal spec annotation or something.

All that said, this doesn't seem to be a terribly important problem to solve 
IMHO.

Regards,

Anthony Liguori

>