Re: [Qemu-devel] [PATCH v2 0/4] aio: experimental virtio-blk polling mode

[Karl Rister]

I think these results look a bit more in line with expectations on the
quick sniff test:

QEMU_AIO_POLL_MAX_NS      IOPs
               unset    26,299
                   1    25,929
                   2    25,753
                   4    27,214
                   8    27,053
                  16    26,861
                  32    24,752
                  64    25,058
                 128    24,732
                 256    25,560
                 512    24,614
               1,024    25,186
               2,048    25,829
               4,096    25,671
               8,192    27,896
              16,384    38,086
              32,768    35,493
              65,536    38,496
             131,072    38,296

I did a spot check of CPU utilization when the polling started having
benefits.

Without polling (QEMU_AIO_POLL_MAX_NS=unset) the iothread's CPU usage
looked like this:

user time:   25.94%
system time: 22.11%

With polling and QEMU_AIO_POLL_MAX_NS=16384 the iothread's CPU usage
looked like this:

user time:   78.92%
system time: 20.80%

Karl

On 11/16/2016 11:46 AM, Stefan Hajnoczi wrote:
> v2:
>  * Uninitialized node->deleted gone [Fam]
>  * Removed 1024 polling loop iteration qemu_clock_get_ns() optimization which
>    created a weird step pattern [Fam]
>  * Unified with AioHandler, dropped AioPollHandler struct [Paolo]
>    (actually I think Paolo had more in mind but this is the first step)
>  * Only poll when all event loop resources support it [Paolo]
>  * Added run_poll_handlers_begin/end trace events for perf analysis
>  * Sorry, Christian, no virtqueue kick suppression yet
> 
> Recent performance investigation work done by Karl Rister shows that the
> guest->host notification takes around 20 us.  This is more than the "overhead"
> of QEMU itself (e.g. block layer).
> 
> One way to avoid the costly exit is to use polling instead of notification.
> The main drawback of polling is that it consumes CPU resources.  In order to
> benefit performance the host must have extra CPU cycles available on physical
> CPUs that aren't used by the guest.
> 
> This is an experimental AioContext polling implementation.  It adds a polling
> callback into the event loop.  Polling functions are implemented for 
> virtio-blk
> virtqueue guest->host kick and Linux AIO completion.
> 
> The QEMU_AIO_POLL_MAX_NS environment variable sets the number of nanoseconds 
> to
> poll before entering the usual blocking poll(2) syscall.  Try setting this
> variable to the time from old request completion to new virtqueue kick.
> 
> By default no polling is done.  The QEMU_AIO_POLL_MAX_NS must be set to get 
> any
> polling!
> 
> Stefan Hajnoczi (4):
>   aio: add AioPollFn and io_poll() interface
>   aio: add polling mode to AioContext
>   virtio: poll virtqueues for new buffers
>   linux-aio: poll ring for completions
> 
>  aio-posix.c                 | 115 
> ++++++++++++++++++++++++++++++++++++++++++--
>  async.c                     |  14 +++++-
>  block/curl.c                |   8 +--
>  block/iscsi.c               |   3 +-
>  block/linux-aio.c           |  19 +++++++-
>  block/nbd-client.c          |   8 +--
>  block/nfs.c                 |   7 +--
>  block/sheepdog.c            |  26 +++++-----
>  block/ssh.c                 |   4 +-
>  block/win32-aio.c           |   4 +-
>  hw/virtio/virtio.c          |  18 ++++++-
>  include/block/aio.h         |   8 ++-
>  iohandler.c                 |   2 +-
>  nbd/server.c                |   9 ++--
>  stubs/set-fd-handler.c      |   1 +
>  tests/test-aio.c            |   4 +-
>  trace-events                |   4 ++
>  util/event_notifier-posix.c |   2 +-
>  18 files changed, 207 insertions(+), 49 deletions(-)
> 


-- 
Karl Rister <address@hidden>