Re: [Qemu-devel] I/O parallelism on QCOW2

[Xingbo Wu]

On Fri, Sep 5, 2014 at 6:02 AM, Stefan Hajnoczi <address@hidden> wrote:

On Thu, Sep 04, 2014 at 12:32:12PM -0400, Xingbo Wu wrote:
>   After running a 16-thread sync-random-write test against qcow2, It is
> observed that QCOW2 seems to be serializing all its metadata-related writes.
>  If qcow2 is designed to do this,* then what is the concern?* What would go
> wrong if this ordering is relaxed?

How do you know that serializing part of the write request is a
significant bottleneck?

Please post your benchmark results with raw, qed, and qcow2 handling 1-,
8-, and 16-threads of I/O (or whatever similar benchmarks you have run).

The bottleneck may actually be something else, so please share your
benchmark configuration and results.

Here is the fio job file:

----j1.fio:

[j1]

direct=1

ioengine=psync

thread

fdatasync=1

runtime=300

numjobs=$NJ

# filename=/dev/sd? for raw disk

filename=/dev/nbd0

rw=write

bs=64k

offset_increment=1G

----EOF

qcow2 image is created on the raw disk with -o lazy_refcounts.

The raw disk is a D3200AAJS

 

As you can see, the test is to measure the performance on synchronize writes.

So practically the result does not mean qcow2 has this bottleneck with real workload.

> By providing less features, raw-file and QED scales well on parallel I/O
> workload. I believe qcow2 does this with clear reasons. Thanks!

QED serializes allocating writes, see qed_aio_write_alloc().

In qcow2 the BdrvQcowState->lock is held across metadata updates.  The
important pieces here are:
 * qcow2_co_writev() only releases the lock around data writes
   (including COW).

Thanks. This is what I want to confirm.

So the lock is held during the metadata write and related I/O acticity? That's why I saw serialized metadata updates in the trace.

Could the lock be released during metadata I/O?

 * qcow2_co_flush_to_os() holds the lock around metadata updates

*flush_to_os moves the data down to the image file, but not necessarily flush them to disk.

This function should usually returns fast with no actual disk I/O. The later calls to flush the image file would incur the FLUSH to disk. Is that correct?

If so, the locking here does not matter.

 

Stefan

--

Cheers!
       吴兴博  Wu, Xingbo <address@hidden>