Re: [Qemu-devel] [PATCH v5 0/9] XBZRLE delta for live migration of large memory apps

[Anthony Liguori]

On 01/03/2012 09:34 AM, Orit Wasserman wrote:
> Changes from v4:
> 1) Rebase
> 2) divide patch into 9 patches
> 3) move memory allocation into cache_insert
>
> By using XBZRLE (Xor Binary Zero Run-Length-Encoding) we can reduce VM downtime
> and total live-migration time of VMs running memory write intensive workloads
> typical of large enterprise applications such as SAP ERP Systems, and generally
> speaking for any application with a sparse memory update pattern.
>
> On the sender side XBZRLE is used as a compact delta encoding of page updates,
> retrieving the old page content from an LRU cache (default size of 64 MB). The
> receiving side uses the existing page content and XBZRLE to decode the new page
> content.
>
> Work was originally based on research results published VEE 2011: Evaluation of
> Delta Compression Techniques for Efficient Live Migration of Large Virtual
> Machines by Benoit, Svard, Tordsson and Elmroth. Additionally the delta encoder
> XBRLE was improved further using XBZRLE instead.
>
> XBZRLE has a sustained bandwidth of 2-2.5 GB/s for typical workloads making it
> ideal for in-line, real-time encoding such as is needed for live-migration.
>
> A typical usage scenario:
>      {qemu} migrate_set_cachesize 256m

How does one intelligently choose a cache size?  How bad is it to choose the 
wrong cache size?

>      {qemu} migrate -x -d tcp:destination.host:4444

We need to make this a negotiated feature, not a requirement for the user to 
figure out.

Please go back to the previous thread for more details on this.

Regards,

Anthony Liguori

>      {qemu} info migrate
>      ...
>      transferred ram-duplicate: A kbytes
>      transferred ram-duplicate: B pages
>      transferred ram-normal: C kbytes
>      transferred ram-normal: D pages
>      transferred ram-xbrle: E kbytes
>      transferred ram-xbrle: F pages
>      overflow ram-xbrle: G pages
>      cache-hit ram-xbrle: H pages
>      cache-lookup ram-xbrle: J pages
>
> Testing: live migration with XBZRLE completed in 110 seconds, without live
> migration was not able to complete.
>
> A simple synthetic memory r/w load generator:
> ..    include<stdlib.h>
> ..    include<stdio.h>
> ..    int main()
> ..    {
> ..        char *buf = (char *) calloc(4096, 4096);
> ..        while (1) {
> ..            int i;
> ..            for (i = 0; i<  4096 * 4; i++) {
> ..                buf[i * 4096 / 4]++;
> ..            }
> ..            printf(".");
> ..        }
> ..    }
>
> Signed-off-by: Benoit Hudzia<address@hidden>
> Signed-off-by: Petter Svard<address@hidden>
> Signed-off-by: Aidan Shribman<address@hidden>
>
>   arch_init.c       |  546 +++++++++++++++++++++++++++++++++++++++++++++++++----
>   block-migration.c |    4 +-
>   hmp-commands.hx   |   34 +++-
>   hw/hw.h           |    4 +-
>   migration.c       |   51 +++++-
>   migration.h       |   19 ++
>   qmp-commands.hx   |   44 ++++-
>   savevm.c          |   11 +-
>   sysemu.h          |    4 +-
>   9 files changed, 653 insertions(+), 64 deletions(-)