Re: [Qemu-devel] [PATCH v3] XBZRLE delta for live migration of large memory apps

[Alexander Graf]

On 02.08.2011, at 15:45, Shribman, Aidan wrote:

> Subject: [PATCH v3] XBZRLE delta for live migration of large memory apps
> From: Aidan Shribman <address@hidden>
> 
> By using XBZRLE (Xor Binary Zero Run-Length-Encoding) we can reduce VM 
> downtime
> and total live-migration time for VMs running memory write intensive workloads
> typical of large enterprise applications such as SAP ERP Systems, and 
> generally
> speaking for representative of any application with a sparse memory update 
> pattern.
> 
> On the sender side XBZRLE is used as a compact delta encoding of page updates,
> retrieving the old page content from an LRU cache (default size of 64 MB). The
> receiving side uses the existing page content and XBZRLE to decode the new 
> page
> content.
> 
> Work was originally based on research results published VEE 2011: Evaluation 
> of
> Delta Compression Techniques for Efficient Live Migration of Large Virtual
> Machines by Benoit, Svard, Tordsson and Elmroth. Additionally the delta 
> encoder
> XBRLE was improved further using XBZRLE instead.
> 
> XBZRLE has a sustained bandwidth of 1.5-2.2 GB/s for typical workloads making 
> it
> ideal for in-line, real-time encoding such as is needed for live-migration.
> 
> A typical usage scenario:
>    {qemu} migrate_set_cachesize 256m
>    {qemu} migrate -x -d tcp:destination.host:4444
>    {qemu} info migrate
>    ...
>    transferred ram-duplicate: A kbytes
>    transferred ram-duplicate: B pages
>    transferred ram-normal: C kbytes
>    transferred ram-normal: D pages
>    transferred ram-xbrle: E kbytes
>    transferred ram-xbrle: F pages
>    overflow ram-xbrle: G pages
>    cache-hit ram-xbrle: H pages
>    cache-lookup ram-xbrle: J pages
> 
> Testing: live migration with XBZRLE completed in 110 seconds, without live
> migration was not able to complete.
> 
> A simple synthetic memory r/w load generator:
> ..    include <stdlib.h>
> ..    include <stdio.h>
> ..    int main()
> ..    {
> ..        char *buf = (char *) calloc(4096, 4096);
> ..        while (1) {
> ..            int i;
> ..            for (i = 0; i < 4096 * 4; i++) {
> ..                buf[i * 4096 / 4]++;
> ..            }
> ..            printf(".");
> ..        }
> ..    }
> 
> Signed-off-by: Benoit Hudzia <address@hidden>
> Signed-off-by: Petter Svard <address@hidden>
> Signed-off-by: Aidan Shribman <address@hidden>


So if I understand correctly, this enabled delta updates for dirty pages? Would 
it be possible to do the same on the block layer, so that VM backing file data 
could potentially save the new information as delta over the old block? 
Especially with metadata updates, that could save quite some disk space.

Of course that would mean that a block is no longer the size of a block :). 
Maybe something to consider for qcow3?


Alex