Re: [PATCH v1 00/11] Convert avocado tests to normal Python unittests

[Daniel P . Berrangé]

On Tue, Jul 16, 2024 at 01:26:03PM +0200, Thomas Huth wrote:
> The Avocado v88 that we use in QEMU is already on a life support
> system: It is not supported by upstream anymore, and with the latest
> versions of Python, it won't work anymore since it depends on the
> "imp" module that has been removed in Python 3.12.
> 
> There have been several attempts to update the test suite in QEMU
> to a newer version of Avocado, but so far no attempt has successfully
> been merged yet.
> 
> Additionally, the whole "make check" test suite in QEMU is using the
> meson test runner nowadays, so running the python-based tests via the
> Avocodo test runner looks and feels quite like an oddball, requiring
> the users to deal with the knowledge of multiple test runners in
> parallel (e.g. the timeout settings work completely differently).
> 
> So instead of trying to update the python-based test suite in QEMU
> to a newer version of Avocado, we should maybe try to better integrate
> it with the meson test runner instead. Indeed most tests work quite
> nicely without the Avocado framework already, as you can see with
> this patch series - it does not convert all tests, just a subset so
> far, but this already proves that many tests only need small modifi-
> cations to work without Avocado.
> 
> Only tests that use the LinuxTest / LinuxDistro and LinuxSSHMixIn
> classes (e.g. based on cloud-init images or using SSH) really depend
> on the Avocado framework, so we'd need a solution for those if we
> want to continue using them. One solution might be to simply use the
> required functions from avocado.utils for these tests, and still run
> them via the meson test runner instead, but that needs some further
> investigation that will be done later.
> 
> 
> Now if you want to try out these patches: Apply the patches, then
> recompile and then run:
> 
>  make check-functional
> 
> You can also run single targets e.g. with:
> 
>  make check-functional-ppc
> 
> You can also run the tests without any test runner now by
> setting the PYTHONPATH environment variable to the "python" folder
> of your source tree, and by specifying the build directory via
> QEMU_BUILD_ROOT (if autodetection fails) and by specifying the
> QEMU binary via QEMU_TEST_QEMU_BINARY. For example:
> 
>  export PYTHONPATH=$HOME/qemu/python
>  export QEMU_TEST_QEMU_BINARY=qemu-system-x86_64
>  export PYTHONPATH=$HOME/qemu/build
>  ~/qemu/tests/functional/test_virtio_version.py

For the whole series as is

 Tested-by: Daniel P. Berrangé <berrange@redhat.com>

as it does what you claim it does here when I tried it.

> The logs of the tests can be found in the build directory under
> tests/functional/<testname> - console log and general logs will
> be put in separate files there.

As an example, one dir name appears to be:

  __main__.MemAddrCheck.test_phybits_ok_pentium_pae

I'd rather prefer it if the dir name matched the test script
file name - in this case test_mem_addr_space.py, as I don't
want to have to lookup which class names were defined inside
each test script. We could drop the "test_" prefix from the
method name too

IOW, could we make this dir name be:

  test_mem_addr_space.phybits_ok_pentium_pae


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|