Re: [Qemu-devel] [Qemu-block] [PATCH 0/6] QTests: use Python to run complex tests

[Stefan Hajnoczi]

On Thu, Dec 14, 2017 at 03:35:07PM +0000, Nir Soffer wrote:
> On Thu, Dec 14, 2017 at 11:39 AM Stefan Hajnoczi <address@hidden>
> wrote:
> 
> > On Wed, Dec 13, 2017 at 06:35:51PM -0300, Philippe Mathieu-Daudé wrote:
> > > Hi,
> > >
> > > With this series we can now write tests using Python rather than C.
> > > For complex tests this can reduce the test development time, we can
> > focus on
> > > the test purposes instead of his implementation details.
> > >
> > > - 1,2: we already have Python classes to run Block tests, move all the
> > >   non Block-related methods to qtest.py,
> > > - 3: default TEST_DIR to TMPDIR,
> > > - 4: add a method to restrict tests to a list of supported machines,
> > > - 5: since the Block tests are output sensitive, do not mess with their
> > >   current tuned iotests::main(unittest), add a more generic one in
> > qtest.py,
> > > - 6: finally modify the tests Makefile to run C/Python tests with the
> > same
> > >   rule.
> >
> > Python tests are great for functional tests of qemu-system-* that
> > interact via the command-line and QMP monitor.  From this perspective I
> > think the series is good.
> >
> > > to have a better idea, here is a snippet from the next series:
> > >
> > >     class TestSdCardSpecV2(qtest.QMPTestCase):
> > >         [...]
> > >         def test_power_on_spec_v2(self):
> > >             self.bus.do_cmd(GO_IDLE_STATE)
> > >             [...]
> > >             # verify Card ID
> > >             data = self.bus.do_cmd(ALL_SEND_CID)
> > >             oid, pnm, psn = struct.unpack(">x2s5sxLxxx", data)
> > >             self.assertEqual(oid, "XY") # QEMU default
> > >             self.assertEqual(pnm, "QEMU!") # QEMU default
> > >             self.assertEqual(psn, 0xdeadbeef) # QEMU default
> >
> > Device qtests are better done in C than Python.  Python is not good at
> > binary I/O
> 
> 
> Why do you think that? what is missing in python 2 for binary io?

C code can reuse existing header files for struct definitions instead
of doing oid, pnm, psn = struct.unpack(">x2s5sxLxxx", data).

In general, Python's model for binary data is a marshalling (copy) model
rather than a type casting (in-place access) model.  It's fine for some
things but programs that deal with a lot of raw data may use third-party
unboxed data primitives like numpy's instead.  I definitely wouldn't say
that binary I/O is a strength of Python.

> > and porting this to Python 3 will be extra work later (Python
> > 2 is set for End-of-Life in 2020, see https://pythonclock.org/).
> >
> 
> You can write today code that work with both python 2 and 3. For binary
> io the key is using io.FileIO and io.BytesIO instead of open() and StringIO
> and CStringIO.

Yes, it's possible to write the code carefully and test under both
Python 2 & 3.  We have to do that for Python code in QEMU since the
world is currently transitioning and both Python versions are in use.

So in this case, where it's questionable to start a new device-level
testing framework in the first place, the extra trouble of dealing with
Python 2 & 3 makes it even less appealing.

Stefan

signature.asc
Description: PGP signature