Re: [PATCH v9 2/3] Acceptance test: add "boot_linux" tests

[Cleber Rosa]

On Thu, Feb 20, 2020 at 01:49:40PM -0300, Wainer dos Santos Moschetta wrote:
> On 2/19/20 11:06 PM, Cleber Rosa wrote:
> > +
> > +    def test_virt_tcg(self):
> > +        """
> > +        :avocado: tags=accel:tcg
> > +        :avocado: tags=cpu:cortex-a53
> > +        """
> > +        if not tcg_available(self.qemu_bin):
> > +            self.cancel(TCG_NOT_AVAILABLE)
> > +        self.vm.add_args("-accel", "tcg")
> > +        self.vm.add_args('-cpu', 'cortex-a53')
> > +        self.add_common_args()
> > +        self.launch_and_wait()
> > +
> > +    def test_virt_kvm(self):
> > +        """
> > +        :avocado: tags=accel:kvm
> > +        :avocado: tags=cpu:host
> > +        """
> > +        if not kvm_available(self.arch, self.qemu_bin):
> > +            self.cancel(KVM_NOT_AVAILABLE)
> > +        self.vm.add_args("-accel", "kvm")
> > +        self.vm.add_args("-cpu", "host")
> > +        self.add_common_args()
> > +        self.launch_and_wait()
> 
> 
> For aarch64 tests it seems '-cpu max' is the best choice. See in
> https://www.mail-archive.com/address@hidden/msg672755.html
> 
>

+drew

Thanks for pointing that out.  There's one thing, though, which I can
not agree on.  And I know that Drew is an expert on the matter, which
makes it harder to disagree on... but, I've got results which clearly
indicate that *not using* the gic-version machine parameter still gets
me KVM:

./tests/venv/bin/avocado run 
tests/acceptance/boot_linux.py:BootLinuxAarch64.test_virt_kvm
JOB ID     : 21a394b884b474ceee0a045b3e74f98da0aee023
JOB LOG    : 
/home/cleber/avocado/job-results/job-2020-02-20T14.28-21a394b/job.log
 (1/1) tests/acceptance/boot_linux.py:BootLinuxAarch64.test_virt_kvm: PASS 
(35.10 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME   : 35.87 s

VM launch command:
   aarch64-softmmu/qemu-system-aarch64
   -display none
   -vga none
   -chardev socket,id=mon,path=/var/tmp/tmpntz_r_h7/qemu-18331-monitor.sock
   -mon chardev=mon,mode=control
   -machine virt
   -chardev 
socket,id=console,path=/var/tmp/tmpntz_r_h7/qemu-18331-console.sock,server,nowait
   -serial chardev:console
   -smp 2
   -m 1024
   -drive 
file=/var/tmp/avocado_u9jm04di/avocado_job_28oth9kk/1-tests_acceptance_boot_linux.py_BootLinuxAarch64.test_virt_kvm/Fedora-Cloud-Base-31-1.9.aarch64-05265df5.qcow2
 -drive 
file=/var/tmp/avocado_u9jm04di/avocado_job_28oth9kk/1-tests_acceptance_boot_linux.py_BootLinuxAarch64.test_virt_kvm/cloudinit.iso,format=raw
   -accel kvm
   -cpu host
   -bios /home/cleber/build/qemu/pc-bios/edk2-aarch64-code.fd
   -device virtio-rng-pci,rng=rng0
   -object rng-random,id=rng0,filename=/dev/urandom

Guest boot messages shows:
[    1.538955] systemd[1]: Detected virtualization kvm.
[    1.539828] systemd[1]: Detected architecture arm64.

This is in contrast with:

./tests/venv/bin/avocado run 
tests/acceptance/boot_linux.py:BootLinuxAarch64.test_virt_tcg 
JOB ID     : 90b9412f700e52428b59e97719496c30b4f54435
JOB LOG    : 
/home/cleber/avocado/job-results/job-2020-02-20T14.32-90b9412/job.log
 (1/1) tests/acceptance/boot_linux.py:BootLinuxAarch64.test_virt_tcg: PASS 
(581.14 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME   : 581.93 s

VM launch command:
   aarch64-softmmu/qemu-system-aarch64
   -display none
   -vga none
   -chardev socket,id=mon,path=/var/tmp/tmpa6i4livg/qemu-18498-monitor.sock
   -mon chardev=mon,mode=control
   -machine virt
   -chardev 
socket,id=console,path=/var/tmp/tmpa6i4livg/qemu-18498-console.sock,server,nowait
   -serial chardev:console
   -smp 2
   -m 1024
   -drive 
file=/var/tmp/avocado_slcj2x9e/avocado_job_x5u__309/1-tests_acceptance_boot_linux.py_BootLinuxAarch64.test_virt_tcg/Fedora-Cloud-Base-31-1.9.aarch64-5b006a2f.qcow2
 -drive 
file=/var/tmp/avocado_slcj2x9e/avocado_job_x5u__309/1-tests_acceptance_boot_linux.py_BootLinuxAarch64.test_virt_tcg/cloudinit.iso,format=raw
   -accel tcg
   -cpu cortex-a53
   -bios /home/cleber/build/qemu/pc-bios/edk2-aarch64-code.fd
   -device virtio-rng-pci,rng=rng0
   -object rng-random,id=rng0,filename=/dev/urandom'

Guest boot messages shows:
[   28.606310] systemd[1]: Detected virtualization qemu.
[   28.607861] systemd[1]: Detected architecture arm64.

And with regards to the CPU type, IIRC, "max" will fallback to the
best CPU on TCG mode.  As a general best practice in testing, I'd
rather not have this dynamic aspect where we can avoid it.  Looks like
with TCG we can set it to one CPU and validate that the guests work on
that configuration.

IIUC, by using either "-cpu host" or "-cpu max" for KVM, we may end up
having the same test PASS or FAIL because of the (dynamic) host CPU.
That's not ideal for testing purposes, but given it's outside of our
control, do best we can do is keep track of the host CPU (via Avocado's
sysinfo collection).

Also, I've used the same CPU model that has been used on
boot_linux_console.py:BootLinuxConsole.test_aarch64_virt, which may be
a plus.

> > +
> > +
> > +class BootLinuxPPC64(BootLinux):
> > +    """
> > +    :avocado: tags=arch:ppc64
> > +    """
> > +
> > +    chksum = 
> > '7c3528b85a3df4b2306e892199a9e1e43f991c506f2cc390dc4efa2026ad2f58'
> > +
> > +    def test_pseries_tcg(self):
> > +        """
> > +        :avocado: tags=machine:pseries
> > +        :avocado: tags=accel:tcg
> > +        """
> > +        if not tcg_available(self.qemu_bin):
> > +            self.cancel(TCG_NOT_AVAILABLE)
> > +        self.vm.add_args("-accel", "tcg")
> > +        self.launch_and_wait()
> > +
> > +
> > +class BootLinuxS390X(BootLinux):
> > +    """
> > +    :avocado: tags=arch:s390x
> > +    """
> > +
> > +    chksum = 
> > '4caaab5a434fd4d1079149a072fdc7891e354f834d355069ca982fdcaf5a122d'
> > +
> > +    def test_s390_ccw_virtio_tcg(self):
> > +        """
> > +        :avocado: tags=machine:s390-ccw-virtio
> > +        :avocado: tags=accel:tcg
> > +        """
> > +        if not tcg_available(self.qemu_bin):
> > +            self.cancel(TCG_NOT_AVAILABLE)
> > +        self.vm.add_args("-accel", "tcg")
> > +        self.launch_and_wait()
> > diff --git a/tests/requirements.txt b/tests/requirements.txt
> > index a2a587223a..a3b5fe4159 100644
> > --- a/tests/requirements.txt
> > +++ b/tests/requirements.txt
> > @@ -1,4 +1,5 @@
> >   # Add Python module requirements, one per line, to be installed
> >   # in the tests/venv Python virtual environment. For more info,
> >   # refer to: https://pip.pypa.io/en/stable/user_guide/#id1
> > -avocado-framework==72.0
> > +avocado-framework==74.0
> > +pycdlib==1.9.0
> 
> 
> Tested on x86_64 machine, the tests behave correctly with following
> configurations:
> 
> 1. ---target-list=x86_64-softmmu --disable-tcg
> 
> 2. ---target-list=x86_64-softmmu --disable-kvm
> 
> 3. --target-list=x86_64-softmmu,aarch64-softmmu,ppc64-softmmu,s390x-softmmu
> 
> But failed if:
> 
> 3. ---target-list=x86_64-softmmu --disable-tools.
> 
> And the error message is:
> 
> (01/32) tests/acceptance/boot_linux.py:BootLinuxX8664.test_pc_i440fx_tcg:
> ERROR: Command 'qemu-img' could not be found in any of the PATH dirs:
> ['/usr/bin', '/usr/sbin', '/usr/lib64/ccache', '/bin', '/root/bin', '/sbin',
> '/usr/local/sbin', '/usr/local/bin', '/usr/libexec'] (1.58 s)

This is what I call comprehensive testing! Thanks!

It looks like I was not paying that much attention to what happens during
the "self.boot.path" attribute access, and had left it "unprotected" in
the QEMU command line arguments assignment.  But there's where a lazy
snapshot image creation is attempted.

I've adjusted the code and will have that on a v10.

> 
> Thanks!
> 
> - Wainer
> 
> 

Thanks a lot for the review!
- Cleber.

signature.asc
Description: PGP signature