Re: [Qemu-devel] [PATCH 0/12 RFC v2] Localhost migration

[Paolo Bonzini]

Il 25/07/2013 22:18, Lei Li ha scritto:
> Hi,
> 
> This patch series tries to add localhost migration support to 
> Qemu. 
> 
> When doing localhost migration, the host memory will balloon
> up during the period, might consume double memories for some time.
> So we want to add a new live migration mechanism localhost
> migration.
> 
> Following I copied from last version that Anthony added for the 
> benefit of the other reviewers:
> 
> The goal here is to allow "live upgrade" of a running QEMU instance.  
> The work flow would look like this:
> 
> 1) Guests are running QEMU release 1.6.1
> 2) Admin installs QEMU release 1.6.2 via RPM or deb
> 3) Admin does localhost migration with page flipping to use new version
>    of QEMU.
> 
> Page flipping is used in order to avoid requiring that there is enough
> free memory to fit an additional copy of the largest guest which is the
> requirement today with localhost migration.
> 
> You can also read from the link below:
> http://lists.gnu.org/archive/html/qemu-devel/2013-06/msg02577.html
> 
> The plan is:
> 
> 1) Add new command to do localhost migration.
> 
>    The qmp interface introduced like:
> 
>    { 'command': 'localhost-migrate', 'data': {'uri': 'str'} }
> 
> 2) Use different mechanism than current live migration.
> 
>    The very basic work flow like:
> 
>        qemu on the source (the source and destination are both on localhost)
>               |
>               V
>            Stop VM
>               |
>               V   
>           Create threads
>               |
>               V
>        Page flipping through vmspice
>               |
>               V
>        MADV_DONTNEED the ram pages which are already flipped
>               |
>               V
>        Migration completes
> 
>    As stopping VM first, we expect/resume the page flipping through vmspice
>    is fast enough to meet *live migration (low downtime).
> 
> Notes:
> Currently the work flow is not exactly the same as description
> above. For the first step, the work flow we implemented is:
> stop VM and copy ram pages via unix domain socket, MADV_DONTNEED 
> ram pages that already copied. After that, will replace to vmsplice
> mechanism instead of copying pages.
> 
> Now it's still a draft version, and as it is implemented separately 
> to the current migration code for a easy start, the next step will 
> be trying to integrate it into the current migration implementation 
> closely. To make sure we are on the right direction that should be 
> headed, please let me know your suggestions on this.
> 
> For the interface, as Anthony has suggested using a flag or a 
> capability, which one would you prefer or any ideas?

Using a capability on the source makes sense; I don't think anything
special is needed on the destination.  The destination just sees a
special packet telling it that a pipe is available via SCM_RIGHTS; then
it fetches the file descriptor and uses it for the new protocol.

It looks like this could reuse a lot of the RAM copying hooks that we
introduced for RDMA.  You shouldn't need to change anything in savevm.c
or arch_init.c

Paolo

> Your comments are very welcome!
> 
> TODO:
> - Integrate to the current implement of migration closely?
> - Introduce a mechanism to exchange a PIPE via SCM_RIGHTS.
> - benchmark/evaluation.
> 
> Lei Li (12):
>   migration: export MIG_STATE_xxx flags 
>   savevm: export qemu_save_device_state()
>   rename is_active to is_block_active
>   arch_init: introduce ram_page_save()
>   arch_init: introduce ram_save_local()
>   arch_init: add save_local_setup to savevm_ram_handlers
>   savevm: introduce qemu_savevm_local()
>   savevm: adjust is_ram check in register_savevm_live()
>   migration-local: implementation of outgoing part
>   migration-local: implementation of incoming part
>   migration-local: add option to command line for local incoming
>   hmp:add hmp_localhost_migration interface
> 
>  Makefile.objs                 |    1 +
>  arch_init.c                   |   110 +++++++++++++++++++
>  block-migration.c             |    2 +-
>  hmp-commands.hx               |   17 ++++
>  hmp.c                         |   13 +++
>  hmp.h                         |    1 +
>  include/migration/migration.h |   32 +++++++
>  include/sysemu/sysemu.h       |    1 +
>  migration-local.c             |  228 
> +++++++++++++++++++++++++++++++++++++++++
>  migration-unix.c              |   60 ++++++++++++
>  migration.c                   |    8 --
>  qapi-schema.json              |   14 +++
>  qemu-options.hx               |    9 ++
>  qmp-commands.hx               |   22 +++++
>  savevm.c                      |   100 +++++++++++++++--
>  vl.c                          |   14 +++
>  16 files changed, 613 insertions(+), 19 deletions(-)
>  create mode 100644 migration-local.c
>