[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] Improving QMP test coverage
From: |
Markus Armbruster |
Subject: |
Re: [Qemu-devel] Improving QMP test coverage |
Date: |
Tue, 18 Jul 2017 18:24:19 +0200 |
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux) |
Stefan Hajnoczi <address@hidden> writes:
> On Thu, Jul 13, 2017 at 05:28:52PM +0200, Markus Armbruster wrote:
>> What can we do to improve QMP testing? Sadly, I don't have the master
>> plan ready. I can tell people their new code needs to come with tests,
>> but that won't help much unless subsystem maintainers pick up the habit,
>> too. There are a few obvious tests to write for old code, such as a
>> generic test of query-like commands without arguments and without side
>> effects, similar to what test-hmp.c does for HMP command info (I hope to
>> get around to that one). But for much of the old code, we don't even
>> know where the test coverage holes are.
>>
>> Ideas anyone?
>
> It makes sense for maintainers to ask that new QMP commands come with
> comprehensive tests.
>
> For me the main question is how to begin? What is the easiest and
> preferred way to write QMP command test cases?
>
> Today the most common approach is a qtest test case that sends commands
> and verifies that the expected response is returned. This approach
> works but we could benefit from a discussion about the alternatives
> (e.g. qemu-iotests style shell scripts with output diffing).
Output testing style delegates checking ouput to diff. I rather like it
when text output is readily available. It is when testing QMP. A
non-trivial example using this style could be useful, as discussing
ideas tends to be more productive when they come with patches.
> If it's easy to write tests then developers will contribute more tests.
Yes. Maintainers can more easily demand them, too.
- [Qemu-devel] Improving QMP test coverage, Markus Armbruster, 2017/07/13
- Re: [Qemu-devel] Improving QMP test coverage, Stefan Hajnoczi, 2017/07/17
- Re: [Qemu-devel] Improving QMP test coverage,
Markus Armbruster <=
- Re: [Qemu-devel] Improving QMP test coverage, Stefan Hajnoczi, 2017/07/21
- Re: [Qemu-devel] Improving QMP test coverage, Cleber Rosa, 2017/07/21
- Re: [Qemu-devel] Improving QMP test coverage, Markus Armbruster, 2017/07/24
- Re: [Qemu-devel] Improving QMP test coverage, Cleber Rosa, 2017/07/25
- Re: [Qemu-devel] Improving QMP test coverage, Markus Armbruster, 2017/07/27
- Re: [Qemu-devel] Improving QMP test coverage, Fam Zheng, 2017/07/27
- Re: [Qemu-devel] Improving QMP test coverage, Fam Zheng, 2017/07/27
- Re: [Qemu-devel] Improving QMP test coverage, Daniel P. Berrange, 2017/07/27
- Re: [Qemu-devel] Improving QMP test coverage, Fam Zheng, 2017/07/27
- Re: [Qemu-devel] Improving QMP test coverage, Daniel P. Berrange, 2017/07/27