Re: [Qemu-devel] qemu + rbd block driver with cache=writeback, is live migration safe ?

[Kevin Wolf]

Am 12.04.2014 um 17:01 hat Alexandre DERUMIER geschrieben:
> Hello,
> 
> I known that qemu live migration with disk with cache=writeback are not safe 
> with storage like nfs,iscsi...
> 
> Is it also true with rbd ?

First of all, in order to avoid misunderstandings, let's be clear that
there are three dimensions for the cache configuration of qemu block
devices. In current versions, they are separately configurable and
cache=writeback really expands to:

    cache.writeback=on,cache.direct=off,cache.no-flush=off

The problematic part of this for live migration is generally not
cache.writeback being enabled, but cache.direct being disabled.

The reason for that is that the destination host will open the image
file immediately, because it needs things like the image size to
correctly initialise the emulated disk devices. Now during the migration
the source keeps working on the image, so if qemu read some metadata on
the destination host, that metadata may be stale by the time that the
migration actually completes.

In order to solve this problem, qemu calls bdrv_invalidate_cache(),
which throws away everything that is cached in qemu so that it is reread
from the image. However, this is ineffective if there are other caches
having stale data, such as the kernel page cache. cache.direct bypasses
the kernel page cache, so this is why it's important in many cases.

rbd does, to my knowledge, not use the kernel page cache, so we're safe
from that part. It does however honour the cache.direct flag when it
decides whether to use its own cache. rbd doesn't implement
bdrv_invalidate_cache() in order to clear that cache when migration
completes.

So the answer to your original question is that it's probably _not_ safe
to use live migration with rbd and cache.direct=off.

> If yes, it is possible to disable manually writeback online with qmp ?

No, such a QMP command doesn't exist, though it would be possible to
implement (for toggling cache.direct, that is; cache.writeback is guest
visible and can therefore only be toggled by the guest).

Kevin