Re: [Qemu-devel] [PATCH] Vmchannel PCI device.

[Daniel P. Berrange]

On Sun, Dec 14, 2008 at 08:03:39PM -0600, Anthony Liguori wrote:
> Daniel P. Berrange wrote:
> >On Sun, Dec 14, 2008 at 04:56:49PM -0600, Anthony Liguori wrote:
> >  
> >>Daniel P. Berrange wrote:
> >>    
> >>>On Sun, Dec 14, 2008 at 01:15:42PM -0600, Anthony Liguori wrote:
> >>> 
> >>>One non-QEMU backend I can see being implemented is a DBus daemon,
> >>>providing a simple bus for RPC calls between guests & host.
> >>>      
> >>The main problem with "external" backends is that they cannot easily 
> >>participate in save/restore or live migration.  If you want to have an 
> >>RPC mechanism, I would suggest implementing the backend in QEMU and 
> >>hooking QEMU up to dbus.  Then you can implement proper save/restore.
> >>    
> >
> >DBus is a general purpose RPC service, which has little-to-no knowledge
> >of the semantics of application services running over it. Simply pushing
> >a backend into QEMU can't magically make sure all the application level
> >state is preserved across save/restore/migrate. For some protocols the
> >only viable option may be to explicitly give the equivalent of -EPIPE 
> >/ POLLHUP to the guest and have it explicitly re-establish connectivity 
> >with the host backend and re-initialize neccessary state if desired
> >  
> 
> In the case of dbus, you actually have a shot of making save/restore 
> transparent.  If you send the RPCs, you can parse the messages in QEMU 
> and know when you have a complete buffer.  You can then dispatch the RPC 
> from QEMU (and BTW, perfect example of security, you want the RPCs to 
> originate from the QEMU process).  When you get the RPC response, you 
> can marshal it and make it available to the guest.
> 
> If you ever have a request or response, you should save the partial 
> results as part of save/restore.  You could use the live feature of 
> savevm to attempt to wait until there are no pending RPCs.  In fact, you 
> have to do this because otherwise, the save/restore would be broken.

This is missing the point I was trying to make. Sure you can parse the
RPC calls and know where the boundaries are so you can ensure a consistent
RPC stream, but that's not the state problem I was trying to describe.

There is state in the higher level application relating to the RPC calls.
So, consider two RPC calls made by say NetworkManager

      - Create FOO()
      - Run FOO.Bar()

We save/restore half-way through the 'Run FOO.Bar()' method, QEMU can see
this and so it replays the interruptted 'Run FOO.Bar()' method call upon 
restore. This is usless because there is nothing that says object 'FOO()'
even exists on the host the VM is being restored on.

Thus for this kind of RPC service I believe is it better not to try and
be transparent in save/restore. Explicitly break the communication channel
and allow the guest VM to re-discover the relative services it wants to
talk with on the host.

> This example is particularly bad for EPIPE.  If the guest sends an RPC, 
> what happens if it gets EPIPE?  Has it been completed?  It would make it 
> very difficult to program for this model.

This is really a question of what semantics the application wants to provide.
DBus is not attempting to provide a guarenteed reliable message delivery
service, and its specification does not require any particular semantics 
upon connection failure. This is already true of the existing TCP based
impl of DBus. So whether in-flight RPCs complete or not is 'undefined',
and thus upon re-connection the client in the guest needs to re-discover
whatever it was talking to and verify what state its in.

A service like AMQP/QPid does provide a guarenteed reliable message stream
with the neccessary protocol handshakes/acknowledgments to be able to 
implement clearly defined semantics upon connection failure. ie the app is 
able to get a guarenteed response as to whether the message has been
delivered or not, even in the face of network comms failure.

Daniel
-- 
|: Red Hat, Engineering, London   -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org  -o-  http://virt-manager.org  -o-  http://ovirt.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-  F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|