Re: [Qemu-devel] [PATCH v5 5/5] qemu-iotests: Add 093 for IO throttling

[Max Reitz]

On 2015-01-26 at 22:03, Fam Zheng wrote:
> On Mon, 01/26 15:45, Max Reitz wrote:
> > On 2015-01-16 at 03:46, Fam Zheng wrote:
> > > This case utilizes qemu-io command "aio_{read,write} -q" to verify the
> > > effectiveness of IO throttling options.
> > >
> > > It's implemented by driving the vm timer from qtest protocol, so the
> > > throttling timers are signaled with determinied time duration. Then we
> > > verify the completed IO requests are within 10% error of bps and iops
> > > limits.
> > >
> > > "null" protocol is used as the disk backend so that no actual disk IO is
> > > performed on host, this will make the blockstats much more
> > > deterministic. Both "null-aio" and "null-co" are covered, which is also
> > > a simple cross validation test for the driver code.
> > >
> > > Signed-off-by: Fam Zheng <address@hidden>
> > > ---
> > >   tests/qemu-iotests/093     | 103 +++++++++++++++++++++++++++++++++++++++++++++
> > >   tests/qemu-iotests/093.out |   5 +++
> > >   tests/qemu-iotests/group   |   1 +
> > >   3 files changed, 109 insertions(+)
> > >   create mode 100755 tests/qemu-iotests/093
> > >   create mode 100644 tests/qemu-iotests/093.out
> > NACK. This literally kills my laptop (I can recover when running this test
> > in tmpfs (for some reason inexplicable to me, since this uses the null block
> > drivers...), but I cannot when running it on my HDD).
> >
> > Would it be possible to use larger requests and smaller iops? (Or just the
> > same request size but smaller bps as well)
> Is it because of CPU or memory? 1000 requests for both read and write seem to
> be overkilling since we are measuring 1000 bps and 10 iops, please try if
> reducing to 100 requests works for you.

Probably memory, since I seem to recall you having the same model as me, 
but I can imagine you having more RAM...

100 requests do not work with 128,000 bps/64 iops/10 seconds (because 
that'd be more than 1 MB of data, whereas 100 requests of 4 kB are of 
course only 400 kB), but the following constellations work:
- 100 requests/128,000 bps/64 iops/1 second
- 100 requests/26,214 bps/8 iops/10 seconds (26,214 is about one tenth 
of 262,144 which is a multiple of 4,096, so the real value will not be 
too far off; the iops are limited by $number_of_requests / 
$duration_in_seconds)

I just hope there won't be some other poor guy for whom even this is too 
much...

Max

> > PS: Feel free to tell me I'm doing something wrong, of course. But just
> > ./check -T -raw -c writethrough 093 just killed my laptop, and simply
> > ./check -raw 093 would have probably killed it, too, if I wouldn't have held
> > down ^C after some seconds (I'm listening to music and that's when it began
> > stuttering...).
> >
> > > diff --git a/tests/qemu-iotests/093 b/tests/qemu-iotests/093
> > > new file mode 100755
> > > index 0000000..d12cc25
> > > --- /dev/null
> > > +++ b/tests/qemu-iotests/093
> > > @@ -0,0 +1,103 @@
> > > +#!/usr/bin/env python
> > > +#
> > > +# Tests for IO throttling
> > > +#
> > > +# Copyright (C) 2015 Red Hat, Inc.
> > > +#
> > > +# This program is free software; you can redistribute it and/or modify
> > > +# it under the terms of the GNU General Public License as published by
> > > +# the Free Software Foundation; either version 2 of the License, or
> > > +# (at your option) any later version.
> > > +#
> > > +# This program is distributed in the hope that it will be useful,
> > > +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> > > +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> > > +# GNU General Public License for more details.
> > > +#
> > > +# You should have received a copy of the GNU General Public License
> > > +# along with this program.  If not, see <http://www.gnu.org/licenses/>.
> > > +#
> > > +
> > > +import iotests
> > > +
> > > +class ThrottleTestCase(iotests.QMPTestCase):
> > > +    test_img = "null-aio://"
> > > +
> > > +    def blockstats(self, device):
> > > +        result = self.vm.qmp("query-blockstats")
> > > +        for r in result['return']:
> > > +            if r['device'] == device:
> > > +                stat = r['stats']
> > > +                return stat['rd_bytes'], stat['rd_operations'], 
> > > stat['wr_bytes'], stat['wr_operations']
> > > +        raise Exception("Device not found for blockstats: %s" % device)
> > > +
> > > +    def setUp(self):
> > > +        self.vm = iotests.VM().add_drive(self.test_img)
> > > +        self.vm.launch()
> > > +
> > > +    def tearDown(self):
> > > +        self.vm.shutdown()
> > > +
> > > +    def do_test_throttle(self, seconds, params):
> > > +        def check_limit(limit, num):
> > > +            # IO throttling algorithm is discrete, allow 10% error so the test
> > > +            # is more
> > "more robust"?
> Yes :)
>
> Fam