Re: [PATCH v3 02/24] tests/functional: Add base classes for the upcoming pytest-based tests

[Daniel P . Berrangé]

On Wed, Jul 31, 2024 at 03:24:35PM +0100, Alex Bennée wrote:
> Daniel P. Berrangé <berrange@redhat.com> writes:
> 
> > From: Thomas Huth <thuth@redhat.com>
> >
> > The file is mostly a copy of the tests/avocado/avocado_qemu/__init__.py
> > file with some adjustments to get rid of the Avocado dependencies (i.e.
> > we also have to drop the LinuxSSHMixIn and LinuxTest for now).
> >
> > The emulator binary and build directory are now passed via
> > environment variables that will be set via meson.build later.
> <snip>
> > diff --git a/tests/functional/qemu_test/testcase.py 
> > b/tests/functional/qemu_test/testcase.py
> > new file mode 100644
> > index 0000000000..82cc1d454f
> > --- /dev/null
> > +++ b/tests/functional/qemu_test/testcase.py
> > @@ -0,0 +1,154 @@
> > +# Test class and utilities for functional tests
> > +#
> > +# Copyright 2018, 2024 Red Hat, Inc.
> > +#
> > +# Original Author (Avocado-based tests):
> > +#  Cleber Rosa <crosa@redhat.com>
> > +#
> > +# Adaption for standalone version:
> > +#  Thomas Huth <thuth@redhat.com>
> > +#
> > +# This work is licensed under the terms of the GNU GPL, version 2 or
> > +# later.  See the COPYING file in the top-level directory.
> > +
> > +import logging
> > +import os
> > +import pycotap
> > +import sys
> > +import unittest
> > +import uuid
> > +
> > +from qemu.machine import QEMUMachine
> > +from qemu.utils import kvm_available, tcg_available
> > +
> > +from .cmd import run_cmd
> > +from .config import BUILD_DIR
> > +
> > +
> > +class QemuBaseTest(unittest.TestCase):
> > +
> > +    qemu_bin = os.getenv('QEMU_TEST_QEMU_BINARY')
> > +    arch = None
> > +
> > +    workdir = None
> > +    log = logging.getLogger('qemu-test')
> > +
> > +    def setUp(self, bin_prefix):
> > +        self.assertIsNotNone(self.qemu_bin, 'QEMU_TEST_QEMU_BINARY must be 
> > set')
> > +        self.arch = self.qemu_bin.split('-')[-1]
> > +
> > +        self.workdir = os.path.join(BUILD_DIR, 'tests/functional', 
> > self.arch,
> > +                                    self.id())
> > +        if not os.path.exists(self.workdir):
> > +            os.makedirs(self.workdir)
> 
> This is racy under --repeat:
> 
>   ==================================== 1/4 
> =====================================
>   test:         qemu:func-quick+func-riscv64 / func-riscv64-riscv_opensbi
>   start time:   14:16:52
>   duration:     0.06s
>   result:       exit status 1
>   command:      
> PYTHONPATH=/home/alex/lsrc/qemu.git/python:/home/alex/lsrc/qemu.git/tests/functional
>  QEMU_BUILD_ROOT=/home/alex/lsrc/qemu.git/builds/all QEMU_TEST_QEMU_BINARY=
>   /home/alex/lsrc/qemu.git/builds/all/qemu-system-riscv64 MALLOC_PERTURB_=71 
> QEMU_TEST_QEMU_IMG=/home/alex/lsrc/qemu.git/builds/all/qemu-img 
> /home/alex/lsrc/qemu.git/builds/al
>   l/pyvenv/bin/python3 
> /home/alex/lsrc/qemu.git/tests/functional/test_riscv_opensbi.py
>   ----------------------------------- stdout 
> -----------------------------------
>   TAP version 13
>   not ok 1 test_riscv_opensbi.RiscvOpenSBI.test_riscv_sifive_u
>   not ok 2 test_riscv_opensbi.RiscvOpenSBI.test_riscv_spike
>   not ok 3 test_riscv_opensbi.RiscvOpenSBI.test_riscv_virt
>   1..3
>   ----------------------------------- stderr 
> -----------------------------------
>   Traceback (most recent call last):
>     File "/home/alex/lsrc/qemu.git/tests/functional/qemu_test/testcase.py", 
> line 85, in setUp
>       super().setUp('qemu-system-')
>     File "/home/alex/lsrc/qemu.git/tests/functional/qemu_test/testcase.py", 
> line 45, in setUp
>       os.makedirs(self.workdir)
>     File "<frozen os>", line 225, in makedirs 
>   FileExistsError: [Errno 17] File exists: 
> '/home/alex/lsrc/qemu.git/builds/all/tests/functional/riscv64/test_riscv_opensbi.RiscvOpenSBI.test_riscv_sifive_u'
> 
>   Traceback (most recent call last):
>     File "/home/alex/lsrc/qemu.git/tests/functional/qemu_test/testcase.py", 
> line 85, in setUp
>       super().setUp('qemu-system-')
>     File "/home/alex/lsrc/qemu.git/tests/functional/qemu_test/testcase.py", 
> line 45, in setUp
>       os.makedirs(self.workdir)
>     File "<frozen os>", line 225, in makedirs 
>   FileExistsError: [Errno 17] File exists: 
> '/home/alex/lsrc/qemu.git/builds/all/tests/functional/riscv64/test_riscv_opensbi.RiscvOpenSBI.test_riscv_spike'
> 
>   Traceback (most recent call last):
>     File "/home/alex/lsrc/qemu.git/tests/functional/qemu_test/testcase.py", 
> line 85, in setUp
>       super().setUp('qemu-system-')
>     File "/home/alex/lsrc/qemu.git/tests/functional/qemu_test/testcase.py", 
> line 45, in setUp
>       os.makedirs(self.workdir)
>     File "<frozen os>", line 225, in makedirs 
>   FileExistsError: [Errno 17] File exists: 
> '/home/alex/lsrc/qemu.git/builds/all/tests/functional/riscv64/test_riscv_opensbi.RiscvOpenSBI.test_riscv_virt'
> 
> 
>   (test program exited with status code 1)
> 
> We could just:
> 
>   os.makedirs(self.workdir, exist_ok = True)

Yeah, that's a better approach.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|