Re: [Qemu-devel] [Qemu-block] [PATCH 0/6] QTests: use Python to run complex tests

[Nir Soffer]

On Thu, Dec 14, 2017 at 11:39 AM Stefan Hajnoczi <address@hidden>
wrote:

> On Wed, Dec 13, 2017 at 06:35:51PM -0300, Philippe Mathieu-Daudé wrote:
> > Hi,
> >
> > With this series we can now write tests using Python rather than C.
> > For complex tests this can reduce the test development time, we can
> focus on
> > the test purposes instead of his implementation details.
> >
> > - 1,2: we already have Python classes to run Block tests, move all the
> >   non Block-related methods to qtest.py,
> > - 3: default TEST_DIR to TMPDIR,
> > - 4: add a method to restrict tests to a list of supported machines,
> > - 5: since the Block tests are output sensitive, do not mess with their
> >   current tuned iotests::main(unittest), add a more generic one in
> qtest.py,
> > - 6: finally modify the tests Makefile to run C/Python tests with the
> same
> >   rule.
>
> Python tests are great for functional tests of qemu-system-* that
> interact via the command-line and QMP monitor.  From this perspective I
> think the series is good.
>
> > to have a better idea, here is a snippet from the next series:
> >
> >     class TestSdCardSpecV2(qtest.QMPTestCase):
> >         [...]
> >         def test_power_on_spec_v2(self):
> >             self.bus.do_cmd(GO_IDLE_STATE)
> >             [...]
> >             # verify Card ID
> >             data = self.bus.do_cmd(ALL_SEND_CID)
> >             oid, pnm, psn = struct.unpack(">x2s5sxLxxx", data)
> >             self.assertEqual(oid, "XY") # QEMU default
> >             self.assertEqual(pnm, "QEMU!") # QEMU default
> >             self.assertEqual(psn, 0xdeadbeef) # QEMU default
>
> Device qtests are better done in C than Python.  Python is not good at
> binary I/O


Why do you think that? what is missing in python 2 for binary io?


> and porting this to Python 3 will be extra work later (Python
> 2 is set for End-of-Life in 2020, see https://pythonclock.org/).
>

You can write today code that work with both python 2 and 3. For binary
io the key is using io.FileIO and io.BytesIO instead of open() and StringIO
and CStringIO.

Nir


>
> More importantly, we already have libqos in C with a guest memory
> allocator, PCI, and virtio support.  Fragmenting the small amount effort
> that goes into device testing will delay libqos reaching critical mass.
> Critical mass is where libqos provides all the infrastructure you need
> to set up a device and focus on your actual test instead of machine,
> bus, or device initialization.  Starting a Python device testing effort
> will just lead to duplication and 2 underdeveloped device testing
> frameworks.
>
> Is there a specific reason why adding SD Card support to libqos is not
> possible in C?
>