[Qemu-devel] [PATCH v4 3/3] dataplane: submit I/O as a batch

[Ming Lei]

Before commit 580b6b2aa2(dataplane: use the QEMU block
layer for I/O), dataplane for virtio-blk submits block
I/O as a batch.

This commit 580b6b2aa2 replaces the custom linux AIO
implementation(including submit I/O as a batch) with QEMU
block layer, but this commit causes ~40% throughput regression
on virtio-blk performance, and removing submitting I/O
as a batch is one of the causes.

This patch applies the newly introduced bdrv_io_plug() and
bdrv_io_unplug() interfaces to support submitting I/O
at batch for Qemu block layer, and in my test, the change
can improve throughput by ~30% with 'aio=native'.

Following my fio test script:

        [global]
        direct=1
        size=4G
        bsrange=4k-4k
        timeout=40
        numjobs=4
        ioengine=libaio
        iodepth=64
        filename=/dev/vdc
        group_reporting=1

        [f]
        rw=randread

Result on one of my small machine(host: x86_64, 2cores, 4thread, guest: 4cores):
        - qemu master: 59K IOPS
        - qemu master with these patches: 81K IOPS
        - 2.0.0 release(dataplane using custom linux aio): 104K IOPS

Reviewed-by: Paolo Bonzini <address@hidden>
Signed-off-by: Ming Lei <address@hidden>
---
 hw/block/dataplane/virtio-blk.c |    5 +++++
 1 file changed, 5 insertions(+)

diff --git a/hw/block/dataplane/virtio-blk.c b/hw/block/dataplane/virtio-blk.c
index c10b7b7..82bb276 100644
--- a/hw/block/dataplane/virtio-blk.c
+++ b/hw/block/dataplane/virtio-blk.c
@@ -201,6 +201,9 @@ static void do_flush_cmd(VirtIOBlockDataPlane *s, 
VirtQueueElement *elem,
     req->elem = elem;
     req->inhdr = inhdr;
 
+    /* flush IOs queued first */
+    bdrv_flush_io_queue(s->blk->conf.bs);
+
     bdrv_aio_flush(s->blk->conf.bs, complete_flush, req);
 }
 
@@ -289,6 +292,7 @@ static void handle_notify(EventNotifier *e)
     int ret;
 
     event_notifier_test_and_clear(&s->host_notifier);
+    bdrv_io_plug(s->blk->conf.bs);
     for (;;) {
         /* Disable guest->host notifies to avoid unnecessary vmexits */
         vring_disable_notification(s->vdev, &s->vring);
@@ -322,6 +326,7 @@ static void handle_notify(EventNotifier *e)
             break;
         }
     }
+    bdrv_io_unplug(s->blk->conf.bs);
 }
 
 /* Context: QEMU global mutex held */
-- 
1.7.9.5