It doesn't make sense to have two classes of flaky tests. While it may
take the constrained environment of CI to trigger failures easily it
doesn't mean they don't occasionally happen on developer machines. As
CI is the gating factor to passing there is no point developers
running the tests locally anyway unless they are trying to fix things.
While we are at it update the language in the docs to discourage the
QEMU_TEST_FLAKY_TESTS becoming a permanent solution.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
---
docs/devel/testing.rst | 31 +++++++++++++++---------
tests/avocado/boot_linux.py | 8 +++---
tests/avocado/boot_linux_console.py | 5 ++--
tests/avocado/intel_iommu.py | 5 ++--
tests/avocado/linux_initrd.py | 5 ++--
tests/avocado/machine_aspeed.py | 8 +++---
tests/avocado/machine_mips_malta.py | 8 +++---
tests/avocado/machine_rx_gdbsim.py | 8 +++---
tests/avocado/machine_s390_ccw_virtio.py | 2 +-
tests/avocado/replay_kernel.py | 5 ++--
tests/avocado/reverse_debugging.py | 14 +++++++----
tests/avocado/smmu.py | 5 ++--
tests/avocado/tuxrun_baselines.py | 4 +--
13 files changed, 67 insertions(+), 41 deletions(-)
diff --git a/tests/avocado/boot_linux.py b/tests/avocado/boot_linux.py
index be30dcbd58..9e9773e6e1 100644
--- a/tests/avocado/boot_linux.py
+++ b/tests/avocado/boot_linux.py
@@ -12,7 +12,7 @@
from avocado_qemu import LinuxTest, BUILD_DIR
-from avocado import skipIf
+from avocado import skipUnless
class BootLinuxX8664(LinuxTest):
@@ -93,7 +93,8 @@ class BootLinuxPPC64(LinuxTest):
timeout = 360
- @skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab')
+ @skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on
GitLab')