Re: [PATCH 06/22] tests/functional: introduce some helpful decorators

[Daniel P . Berrangé]

On Mon, Dec 02, 2024 at 09:27:34AM +0100, Thomas Huth wrote:
> On 29/11/2024 18.31, Daniel P. Berrangé wrote:
> > Reduce repeated boilerplate with some helper decorators:
> > 
> >   @skipIfNotPlatform("x86_64", "aarch64")
> > 
> >    => Skip unless the build host platform matches
> > 
> >   @skipIfMissingCommands("mkisofs", "losetup")
> > 
> >    => Skips unless all listed commands are found in $PATH
> > 
> >   @skipIfMissingImports("numpy", "cv2")
> > 
> >    => Skips unless all listed modules can be imported
> > 
> >   @skipFlakyTest("https://gitlab.com/qemu-project/qemu/-/issues/NNN";)
> > 
> >    => Skips unless env var requests flaky tests with the
> >       reason documented in the referenced gitlab bug
> > 
> >   @skipBigData
> > 
> >    => Skips unless env var permits tests creating big data files
> > 
> >   @skipUntrustedTest
> > 
> >    => Skips unless env var permits tests which are potentially
> >       dangerous to the host


> > +'''
> > +Decorator to skip execution of tests which need large
> > +data storage on the host, unless the
> > +$QEMU_TEST_ALLOW_LARGE_STORAGE env var is set
> 
> Maybe we should also provide some direction what is meant with large
> storage. I've seen some tests that are skipped since they create a disk file
> with 128 MiB. And others are always executed though they create a disk file
> with 512 MiB or even more. What would be a good recommendation here?
> (My gut feeling is maybe ~ 1 GiB? Or better less?)

We're quite limited on disk space in the FreeBSD CI runner, so if we
take account of fact that tests can run in parallel, we definitely want
to err on the smaller side, while maximising coverage available by
default. 128 MB is too small, too many tests would get excluded. I
guess 512MB - 1GB is probably the rough range we should give guidance
for.


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|