qemu-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Qemu-devel] [PATCH v5 0/3] Add common QEMU control functionality to


From: Jeff Cody
Subject: Re: [Qemu-devel] [PATCH v5 0/3] Add common QEMU control functionality to qemu-iotests
Date: Mon, 5 May 2014 11:44:55 -0400
User-agent: Mutt/1.5.21 (2010-09-15)

On Mon, May 05, 2014 at 05:32:09PM +0200, Kevin Wolf wrote:
> Am 05.05.2014 um 17:21 hat Stefan Hajnoczi geschrieben:
> > On Wed, Apr 30, 2014 at 10:55:07AM -0400, Jeff Cody wrote:
> > > This adds some common functionality to control QEMU for qemu-iotests.
> > > 
> > > Additionally, test 085 is updated to use this new functionality.
> > > 
> > > Some minor fixups along the way, to clear up spaced pathname issues, 
> > > for common.rc, test 019, and test 086.
> > > 
> > > 
> > > Jeff Cody (3):
> > >   block: qemu-iotests - add common.qemu, for bash-controlled qemu tests
> > 
> > Once a test launches QEMU, it soon needs to parse QMP commands or wait
> > for QMP events.  That doesn't lend itself to the traditional
> > qemu-iotests shell model.  That is why iotests.py exists.
> > 
> > Shell script is a poor language for test cases that go beyond
> > pre-defined commands whose output is saved for diffing.  The string
> > manipulation is clumsy, JSON is not supported, tricks with fifos can
> > easily deadlock or break when a process terminates unexpectedly, etc.
> > 
> > If we go further in the direction of this patch series, we'll duplicate
> > existing iotests.py code and have complex shell tests that are hard to
> > extend.  I think it's time to draw the line and convert any test cases
> > that need to complexity to Python.
> > 
> > Why not use iotests.py?
> 
> Because it's hard to use. The "compare against reference output" thing
> is the first thing that you lose with iotests.py, and it's the most
> useful feature in qemu-iotests.
> 
> When a Python test case fails, you get into real debugging. When a shell
> script test case fails, you usually see immediately from the reference
> output diff what's wrong.
> 
> I accept iotest.py for anything that needs to evaluate QMP return
> values, reluctantly, because we have nothing better. But that's it, I
> don't actually _like_ using it.
>

I agree with Kevin.  

Also, for a test framework, having some language duplication is not
necessarily a *bad* thing.  One, functionality may be tested in
slightly different ways, due to architectural differences.  Second, it
gives developers increased flexibility in writing test cases in a
language they may be more comfortable in - and that hopefully leads to
an increased number of tests being written.

And if you look at the actual tests (091 and 085, in this case), the
framework is pretty simple to use. If you run into something
unsupported (json parsing, etc.), then you can use iotests.py.

Jeff



reply via email to

[Prev in Thread] Current Thread [Next in Thread]