Re: [Qemu-devel] [PATCH v3] XBZRLE delta for live migration of large memory apps

[Shribman, Aidan]

> From: Anthony Liguori [mailto:address@hidden 
> Sent: Tuesday, August 02, 2011 6:01 PM
> To: Alexander Graf
> Cc: Shribman, Aidan; Kevin Wolf; Stefan Hajnoczi; qemu-devel 
> Developers
> Subject: Re: [Qemu-devel] [PATCH v3] XBZRLE delta for live 
> migration of large memory apps
> 
> On 08/02/2011 09:01 AM, Alexander Graf wrote:
> >
> >>
> >> XBZRLE has a sustained bandwidth of 1.5-2.2 GB/s for 
> typical workloads making it
> >> ideal for in-line, real-time encoding such as is needed 
> for live-migration.
> 
> How does this compare to just doing gzip compression for the 
> same workload?
> 
> Regards,
> 
> Anthony Liguori

For the live-migration case using a synthetic benchmark (which represents the 
discussed enterprise workload) you can see that zlib (59-92 MB/s), lzo 
(418-1123 MB/s) and snappy (372-1362 MB/s) compressions are much slower than 
xbzrle delta compression (1778-2286 MB/s) in addition to having a much bigger 
variance in results for various scenarios.

Aidan

==========================================================
Scenario SPARSE with diff segment of step 1111 len 12
==========================================================
zlib: ENC{1.08s   59 MB/s 2.36%} DEC{0.24s  269 MB/s 100.00%} .. ok
xbzlib: ENC{1.14s   56 MB/s 2.62%} DEC{0.19s  335 MB/s 100.00%} .. ok
lzo: ENC{0.06s 1123 MB/s 2.82%} DEC{0.03s 2286 MB/s 100.00%} .. ok
xblzo: ENC{0.06s 1067 MB/s 2.82%} DEC{0.05s 1391 MB/s 100.00%} .. ok
snappy: ENC{0.05s 1362 MB/s 6.14%} DEC{0.04s 1641 MB/s 100.00%} .. ok
xbsnappy: ENC{0.07s  985 MB/s 6.14%} DEC{0.06s 1103 MB/s 100.00%} .. ok
xbrle: ENC{0.29s  218 MB/s 3.08%} DEC{0.10s  627 MB/s 100.00%} .. ok
xbzrle: ENC{0.03s 2286 MB/s 3.55%} DEC{0.03s 2560 MB/s 100.00%} .. ok

==========================================================
Scenario MEDIUM with diff segment of step 701 len 33
==========================================================
zlib: ENC{0.70s   92 MB/s 5.68%} DEC{0.26s  244 MB/s 100.00%} .. ok
xbzlib: ENC{0.81s   79 MB/s 6.08%} DEC{0.21s  298 MB/s 100.00%} .. ok
lzo: ENC{0.07s  955 MB/s 7.07%} DEC{0.03s 2207 MB/s 100.00%} .. ok
xblzo: ENC{0.07s  865 MB/s 6.34%} DEC{0.05s 1362 MB/s 100.00%} .. ok
snappy: ENC{0.06s 1049 MB/s 9.80%} DEC{0.04s 1641 MB/s 100.00%} .. ok
xbsnappy: ENC{0.07s  914 MB/s 9.27%} DEC{0.06s 1164 MB/s 100.00%} .. ok
xbrle: ENC{0.28s  228 MB/s 10.31%} DEC{0.11s  561 MB/s 100.00%} .. ok
xbzrle: ENC{0.03s 2065 MB/s 8.37%} DEC{0.02s 3368 MB/s 100.00%} .. ok

==========================================================
Scenario DENSE with diff segment of step 203 len 41
==========================================================
zlib: ENC{1.54s   42 MB/s 20.76%} DEC{0.42s  151 MB/s 100.00%} .. ok
xbzlib: ENC{1.39s   46 MB/s 18.68%} DEC{0.41s  156 MB/s 100.00%} .. ok
lzo: ENC{0.11s  561 MB/s 25.58%} DEC{0.03s 2133 MB/s 100.00%} .. ok
xblzo: ENC{0.11s  561 MB/s 21.37%} DEC{0.07s  928 MB/s 100.00%} .. ok
snappy: ENC{0.12s  516 MB/s 25.87%} DEC{0.04s 1561 MB/s 100.00%} .. ok
xbsnappy: ENC{0.15s  435 MB/s 22.80%} DEC{0.07s  928 MB/s 100.00%} .. ok
xbrle: ENC{0.30s  211 MB/s 41.44%} DEC{0.11s  587 MB/s 100.00%} .. ok
xbzrle: ENC{0.04s 1778 MB/s 31.92%} DEC{0.03s 2560 MB/s 100.00%} .. ok

==========================================================
Scenario VERY-DENSE with diff segment of step 121 len 43
==========================================================
zlib: ENC{1.90s   34 MB/s 36.43%} DEC{0.49s  130 MB/s 100.00%} .. ok
xbzlib: ENC{1.60s   40 MB/s 26.71%} DEC{0.48s  132 MB/s 100.00%} .. ok
lzo: ENC{0.15s  418 MB/s 43.34%} DEC{0.03s 2133 MB/s 100.00%} .. ok
xblzo: ENC{0.17s  386 MB/s 32.29%} DEC{0.08s  800 MB/s 100.00%} .. ok
snappy: ENC{0.17s  372 MB/s 41.86%} DEC{0.04s 1488 MB/s 100.00%} .. ok
xbsnappy: ENC{0.21s  305 MB/s 33.46%} DEC{0.08s  762 MB/s 100.00%} .. ok
xbrle: ENC{0.29s  217 MB/s 72.78%} DEC{0.14s  451 MB/s 100.00%} .. ok
xbzrle: ENC{0.03s 2207 MB/s 54.92%} DEC{0.03s 2133 MB/s 100.00%} .. ok