Re: [PATCH for-10.0] tests/functional: Convert the intel_iommu avocado test

[Daniel P . Berrangé]

On Mon, Dec 09, 2024 at 05:31:37PM +0100, Philippe Mathieu-Daudé wrote:
> On 9/12/24 10:09, Daniel P. Berrangé wrote:
> > On Mon, Dec 09, 2024 at 09:26:21AM +0100, Thomas Huth wrote:
> > > On 09/12/2024 09.12, Eric Auger wrote:
> > > > Hi Thomas,
> > > > 
> > > > On 12/6/24 19:17, Thomas Huth wrote:
> > > > > Convert the intel_iommu test to the new functional framework.
> > > > > This test needs some changes since we neither support the old 
> > > > > 'LinuxTest'
> > > > > class in the functional framework yet, nor a way to use SSH for 
> > > > > running
> > > > > commands in the guest. So we now directly download a Fedora kernel and
> > > > > initrd and set up the serial console for executing the commands and 
> > > > > for
> > > > > looking for the results.
> > > > > 
> > > > > Signed-off-by: Thomas Huth <thuth@redhat.com>
> > > > > ---
> > > > >    MAINTAINERS                                   |   1 +
> > > > >    tests/functional/meson.build                  |   1 +
> > > > >    .../test_intel_iommu.py}                      | 119 
> > > > > ++++++++----------
> > > > >    3 files changed, 51 insertions(+), 70 deletions(-)
> > > > >    rename tests/{avocado/intel_iommu.py => 
> > > > > functional/test_intel_iommu.py} (41%)
> > > > >    mode change 100644 => 100755
> 
> 
> > > > > -        self.launch_and_wait()
> > > > > -        self.ssh_command('cat /proc/cmdline')
> > > > > -        self.ssh_command('dmesg | grep -e DMAR -e IOMMU')
> > > > > -        self.ssh_command('find /sys/kernel/iommu_groups/ -type l')
> > > > > -        self.ssh_command('dnf -y install numactl-devel')
> > > > I understand you cannot use ssh yet but the bulk of the test was the
> > > > execution of the dnf install meaning we lose the main substance of it
> > > > through the conversion.
> > > 
> > > Ah, I see, so this was exercising the virtio-net device with the IOMMU ...
> > > and I already wondered why there was this "dnf install" at the end without
> > > doing anything with  the numactl-devel package ... (a comment would have
> > > been helpful here)
> > 
> > FYI, I find 'dnf instal' to be a *highly* undesirable thing todo in
> > our test functional. Its performance is highly non-deterministic
> > depending on what mirror you happen to get sent to, such that it could
> > easily push us over the timeouts. It is also susceptible to periodic
> > broken mirrors, and instability around time of Fefdora EOL. I can't
> > remember if it was this test case, or a different one, but I've seen
> > problems before in avocado with 'dnf install'.
> > 
> > If we want to test working networking, then can we arrange for something
> > more simple & targetted to run, with better worst case performance.
> 
> Could we use 2 virtio-net interfaces inter-connected and stress with
> https://linux.die.net/man/1/ab ?

Do we actually need to stress this ? IMHO for a functional tests we just
need to prove that the device is working at a fairly basic level.
"wget example.com"

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|