[Qemu-block] [PATCH 00/13] throttle: Allow I/O bursts for a user-defined period of time

[Alberto Garcia]

Hello everyone,

the current throttling code in QEMU allows limiting the I/O rate on
block devices. Limits can be set in operations per second (IOPS) or
bytes per second, allowing separate limits for read and write
operations on both cases.

In its basic usage the user can set a limit of, say, 100 IOPS on a
block device passing this option to -drive:

  throttling.iops-total=100

In addition to that, QEMU can also allow the user to do I/O bursts
exceeding that limit up to a configurable rate:

  throttling.iops-total=100,throttling.iops-total-max=2000

With this, the user can do a burst of 2000 IOPS before they're
throttled down to 100 IOPS. Then, after a sufficiently long period of
unused I/O they will be able to do a burst again.

This patch series introduces the possibility to do bursts for longer
period of times. A new setting called throttling.iops-total-max-length
is used to define for how long bursts can be sustained.

So adding throttling.iops-total-max-length=60 to the previous
configuration allows the user to do I/O at a rate of 2000 IOPS for 1
minute before going down to the base rate of 100 IOPS.

This is essentially the same as described in this AWS blog post:

   https://aws.amazon.com/blogs/aws/new-ssd-backed-elastic-block-storage/

As described in the article, a use case for this feature is to allow
better performance when booting the OS or restarting a service while
keeping the average I/O limits lower the rest of the time.

Comments:

 - There are 6 different settings for setting I/O limits: iops-total,
   iops-read, iops-write, bps-total, bps-read, bps-write. This series
   adds one new setting to set the length for each one of those.

   I don't know if there's a good use case that requires such
   fine-grained control. It's of course also possible to make it
   simpler and add just one 'burst-length' setting that would apply
   for all cases, but the current solution is IMHO simple enough and
   consistent with the current API, and if we need to extend it later
   the result is probably going to be ugly.

 - With this series we set "a maximum of X operations/second for a
   period of T seconds". If would also be possible to make it "a
   maximum of X operations/second up to a total of Y operations". It
   would be equivalent (Y = X * T) but I thought the current proposal
   makes a more clear API.

And I think that's all.

Thanks!

Berto

Alberto Garcia (13):
  throttle: Make throttle_compute_timer() static
  throttle: Make throttle_conflicting() set errp
  throttle: Make throttle_max_is_missing_limit() set errp
  throttle: Make throttle_is_valid() set errp
  throttle: Set always an average value when setting a maximum value
  throttle: Merge all functions that check the configuration into one
  throttle: Use throttle_config_init() to initialize ThrottleConfig
  throttle: Add support for burst periods
  throttle: Add command-line settings to define the burst periods
  qapi: Add burst length parameters to block_set_io_throttle
  qapi: Add burst length fields to BlockDeviceInfo
  throttle: Check that burst_level leaks correctly
  throttle: Test throttle_compute_wait() during bursts

 block/qapi.c            |  20 ++++++++
 blockdev.c              |  99 ++++++++++++++++++++++++++----------
 hmp.c                   |  12 +++++
 include/qemu/throttle.h |  55 ++++++++++++++------
 qapi/block-core.json    |  90 ++++++++++++++++++++++++++++-----
 tests/test-throttle.c   |  88 ++++++++++++++++++++++++--------
 util/throttle.c         | 132 ++++++++++++++++++++++++++++++++----------------
 7 files changed, 377 insertions(+), 119 deletions(-)

-- 
2.7.0