Re: [Qemu-devel] [RFC] Disk integrity in QEMU

[Avi Kivity]

Anthony Liguori wrote:
> > If I focus on the sentence "The I/O is synchronous, that is, at
> > the completion of a read(2) or write(2), data is guaranteed to have
> > been transferred. ",
>
> It's extremely important to understand what the guarantee is.  The 
> guarantee is that upon completion on write(), the data will have been 
> reported as written by the underlying storage subsystem.  This does 
> *not* mean that the data is on disk.

It means that as far as the block-io layer of the kernel is concerned, 
the guarantee is met.  If the writes go to to a ramdisk, or to an IDE 
drive with write-back cache enabled, or to disk with write-back cache 
disabled but without redundancy, or to a high-end storage array with 
double-parity protection but without a continuous data protection 
offsite solution, things may still go wrong.

It is up to qemu to provide a strong link in the data reliability chain, 
not to ensure that the entire chain is perfect.  That's up to the 
administrator or builder of the system.

> If you have a normal laptop, your disk has a cache.  That cache does 
> not have a battery backup.  Under normal operations, the cache is 
> acting in write-back mode and when you do a write, the disk will 
> report the write as completed even though it is not actually on disk.  
> If you really care about the data being on disk, you have to either 
> use a disk with a battery backed cache (much more expensive) or enable 
> write-through caching (will significantly reduce performance).

I think that with SATA NCQ, this is no longer true.  The drive will 
report the write complete when it is on disk, and utilize multiple 
outstanding requests to get coalescing and reordering.  Not sure about 
this, though -- some drives may still be lying.

> In the case of KVM, even using write-back caching with the host page 
> cache, we are still honoring the guarantee of O_DIRECT.  We just have 
> another level of caching that happens to be write-back.

No, we are lying.  That's fine if the user tells us to lie, but not 
otherwise.

> > I think there a bug here. If I open a
> > file with the O_DIRECT flag and the host reports back to me that
> > the transfer has completed when in fact its still in the host cache,
> > its a bug as it violates the open()/write() call and there is no
> > guarantee that the data will actually be written.
>
> This is very important, O_DIRECT does *not* guarantee that data 
> actually resides on disk.  There are many possibly places that it can 
> be cached (in the storage controller, in the disks themselves, in a 
> RAID controller).

O_DIRECT guarantees that the kernel is not the weak link in the 
reliability chain.

> > So I guess the real issue isn't what the default should be (although
> > the performance team at Red Hat would vote for cache=off),
>
> The consensus so far has been that we want to still use the host page 
> cache but use it in write-through mode.  This would mean that the 
> guest would only see data completion when the host's storage subsystem 
> reports the write as having completed.  This is not the same as 
> cache=off but I think gives the real effect that is desired.

I am fine with write-through as default, but cache=off should be a 
supported option.

> Do you have another argument for using cache=off?

Performance.

--
error compiling committee.c: too many arguments to function