Re: [Qemu-devel] Adding a persistent writeback cache to qemu

[Alex Bligh]

Sage,

--On 20 June 2013 08:58:19 -0700 Sage Weil <address@hidden> wrote:

> > I'd like to hear from Ceph folks what their position on kernel rbd vs
> > librados is.  Why one do they recommend for QEMU guests and what are the
> > pros/cons?
>
> I agree that a flashcache/bcache-like persistent cache would be a big win
> for qemu + rbd users.

Great.

I think Stefan was really after testing my received wisdom that
ceph+librbd will be greater performance than ceph+blkdev+kernelrbd
(even without the persistent cache), and if so why.

> There are few important issues with librbd vs kernel rbd:
>
>  * librbd tends to get new features more quickly that the kernel rbd
>    (although now that layering has landed in 3.10 this will be less
>    painful than it was).
>
>  * Using kernel rbd means users need bleeding edge kernels, a non-starter
>    for many orgs that are still running things like RHEL.  Bug fixes are
>    difficult to roll out, etc.
>
>  * librbd has an in-memory cache that behaves similar to an HDD's cache
>    (e.g., it forces writeback on flush).  This improves performance
>    significantly for many workloads.  Of course, having a bcache-like
>    layer mitigates this..
>
> I'm not really sure what the best path forward is.  Putting the
> functionality in qemu would benefit lots of other storage backends,
> putting it in librbd would capture various other librbd users (xen, tgt,
> and future users like hyper-v), and using new kernels works today but
> creates a lot of friction for operations.

To be honest I'd not even thought of putting it in librbd (which might
be simpler). I suspect it might be easier to get patches into librbd
than into qemu, and that ensuring cache coherency might be simpler.
If I get time to look at this, would you be interested in taking patches
for this?

--
Alex Bligh