[Qemu-devel] Re: [RFC PATCH 00/20] Kemari for KVM v0.1

[Yoshiaki Tamura]

Anthony Liguori wrote:
> On 04/21/2010 12:57 AM, Yoshiaki Tamura wrote:
> > Hi all,
> >
> > We have been implementing the prototype of Kemari for KVM, and we're
> > sending
> > this message to share what we have now and TODO lists. Hopefully, we
> > would like
> > to get early feedback to keep us in the right direction. Although
> > advanced
> > approaches in the TODO lists are fascinating, we would like to run
> > this project
> > step by step while absorbing comments from the community. The current
> > code is
> > based on qemu-kvm.git 2b644fd0e737407133c88054ba498e772ce01f27.
> >
> > For those who are new to Kemari for KVM, please take a look at the
> > following RFC which we posted last year.
> >
> > http://www.mail-archive.com/address@hidden/msg25022.html
> >
> > The transmission/transaction protocol, and most of the control logic is
> > implemented in QEMU. However, we needed a hack in KVM to prevent rip from
> > proceeding before synchronizing VMs. It may also need some plumbing in
> > the
> > kernel side to guarantee replayability of certain events and
> > instructions,
> > integrate the RAS capabilities of newer x86 hardware with the HA
> > stack, as well
> > as for optimization purposes, for example.
> >
> > Before going into details, we would like to show how Kemari looks. We
> > prepared
> > a demonstration video at the following location. For those who are not
> > interested in the code, please take a look.
> > The demonstration scenario is,
> >
> > 1. Play with a guest VM that has virtio-blk and virtio-net.
> > # The guest image should be a NFS/SAN.
> > 2. Start Kemari to synchronize the VM by running the following command
> > in QEMU.
> > Just add "-k" option to usual migrate command.
> > migrate -d -k tcp:192.168.0.20:4444
> > 3. Check the status by calling info migrate.
> > 4. Go back to the VM to play chess animation.
> > 5. Kill the the VM. (VNC client also disappears)
> > 6. Press "c" to continue the VM on the other host.
> > 7. Bring up the VNC client (Sorry, it pops outside of video capture.)
> > 8. Confirm that the chess animation ends, browser works fine, then
> > shutdown.
> >
> > http://www.osrg.net/kemari/download/kemari-kvm-fc11.mov
> >
> > The repository contains all patches we're sending with this message.
> > For those
> > who want to try, pull the following repository. At running configure,
> > please
> > put --enable-ft-mode. Also you need to apply a patch attached at the
> > end of
> > this message to your KVM.
> >
> > git://kemari.git.sourceforge.net/gitroot/kemari/kemari
> >
> > In addition to usual migrate environment and command, add "-k" to run.
> >
> > The patch set consists of following components.
> >
> > - bit-based dirty bitmap. (I have posted v4 for upstream QEMU on April
> > 2o)
> > - writev() support to QEMUFile and FdMigrationState.
> > - FT transaction sender/receiver
> > - event tap that triggers FT transaction.
> > - virtio-blk, virtio-net support for event tap.
>
> This series looks quite nice!

Thanks for your kind words!

> I think it would make sense to separate out the things that are actually
> optimizations (like the dirty bitmap changes and the writev/readv
> changes) and to attempt to justify them with actual performance data.

I agree with the separation plan.

For dirty bitmap change, Avi and I discussed on patchset for upsream QEMU while 
you were offline (Sorry, if I was wrong).  Could you also take a look?

http://lists.gnu.org/archive/html/qemu-devel/2010-04/msg01396.html

Regarding writev, I agree that it should be backed with actual data, otherwise 
it should be removed.  We attemped to do everything that may reduce the overhead 
of the transaction.

> I'd prefer not to modify the live migration protocol ABI and it doesn't
> seem to be necessary if we're willing to add options to the -incoming
> flag. We also want to be a bit more generic with respect to IO.

I totally agree with your approach not to change the protocol ABI.  Can we add 
an option to -incoming?  Like, -incoming ft_mode, for example
Regarding the IO, let me reply to the next message.

> Otherwise, the series looks very close to being mergable.

Thank you for your comment on each patch.

To be honest, I wasn't that confident because I'm a newbie to KVM/QEMU and 
struggled for how to implement in an acceptable way.

Thanks,

Yoshi

> Regards,
>
> Anthony Liguori