Re: [RFC PATCH 4/8] tests/pytest: add pytest to the meson build system

[Daniel P . Berrangé]

On Fri, Jul 12, 2024 at 12:14:45PM +0200, Thomas Huth wrote:
> On 12/07/2024 11.01, Daniel P. Berrangé wrote:
> > On Thu, Jul 11, 2024 at 01:55:42PM +0200, Thomas Huth wrote:
> > > From: Ani Sinha <ani@anisinha.ca>
> > > 
> > > Integrate the pytest framework with the meson build system. This
> > > will make meson run all the pytests under the pytest directory.
> > 
> > Lets add a note about the compelling benefit of this new approach
> > 
> >    With this change, each functional test becomes subject
> >    to an individual execution timeout, defaulting to 60
> >    seconds, but overridable per-test.
> 
> The avocado runner uses timeouts, too, so it's not really an additional
> benefit that we get here.
> 
> > For CI purposes we'll need to add 'python3-pytest' to
> > tests/lcitool/projects/qemu.yml, and re-generate the
> > the dockerfiles. Some of the other non-gitlab CI
> > integrations probably need manual additions of pytest
> > packages.
> 
> I'm currently rather looking into getting rid of pytest and to use pycotap
> instead: Using the TAP protocol for running the tests, you get a much nicer
> output from the meson test runner, which can then count the subtests and
> properly report SKIPs for tests that have not been run.

I've just looked at pycotap and IIUC, there's no command line
tool equivalent to '/usr/bin/pytest' at all. Each test case
is expected to provide a stub for "__main__" to invoke the
tests. As such each individual test is directly executable.
This meshes nicely with what I'd suggested as changes in
patch 1, and eliminating the intermediate runner process is
a nice further simplification. So I'll be interested to see
your next version using pycotap.


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|