Re: [PATCH 00/12] Introduce new acpi/smbios qtests using biosbits

[Ani Sinha]

On Mon, Jun 27, 2022 at 12:22 PM Ani Sinha <ani@anisinha.ca> wrote:
>
> On Mon, Jun 27, 2022 at 12:14 PM Ani Sinha <ani@anisinha.ca> wrote:
> >
> > Biosbits is a software written by Josh Triplett that can be downloaded by
> > visiting https://biosbits.org/. The github codebase can be found here:
> > https://github.com/biosbits/bits/tree/master. It is a software that 
> > exercizes
> > the bios components such as acpi and smbios tables directly through acpica
> > bios interpreter (a freely available C based library written by Intel,
> > downloadable from https://acpica.org/ and is included with biosbits) 
> > without an
> > operating system getting involved in between.
> > There are several advantages to directly testing the bios in a real physical
> > machine or VM as opposed to indirectly discovering bios issues through the
> > operating system. For one thing, the OSes tend to hide bios problems from 
> > the
> > end user. The other is that we have more control of what we wanted to test
> > and how by directly using acpica interpreter on top of the bios on a running
> > system. More details on the inspiration for developing biosbits and its real
> > life uses can be found in (a) and (b).
> > This patchset contains QEMU qtests written in python that exercizes the QEMU
> > bios components using biosbits and reports test failures.
> >
> > Details of each of the files added by this patchset are provided in the 
> > README
> > file which is part of Patch 11. Every effort to contact Josh, through 
> > various
> > means including email, twitter, linkedIn etc has failed. Hence, the changes 
> > to
> > build biosbits with the newer compiler, upgrade acpica and other changes are
> > currently maintained in a forked project in my personal github. We may want 
> > to
> > maintain bits in a separate fork in a stable repository that is accessible 
> > by
> > QEMU developers.
> >
> > The newly introduced qtest currently only run for x86_64 platform. They pass
> > both when running make check on a baremetal box as well as from inside a vm.
> >
> > Thanks to Igor M for pointing me to this work.
> >
> > (a) 
> > https://blog.linuxplumbersconf.org/2011/ocw/system/presentations/867/original/bits.pdf
> > (b) https://www.youtube.com/watch?v=36QIepyUuhg
>
> Doh, sending the patch series failed since google won't allow file
> sizes larger than 25 MB. The bits prebuilt binaries have sizes 30 MB
> and 40 MB in two separate patches :(

I have pushed the patchset here:
https://gitlab.com/anisinha/qemu/-/commits/acpi-bits .

>
> >
> > Ani Sinha (12):
> >   qtest: meson.build changes required to integrate python based qtests
> >   acpi/tests/bits: add prebuilt bios bits zip archive
> >   acpi/tests/bits: add prebuilt bits generated grub modules and scripts
> >   acpi/tests/bits: initial commit of test scripts that are run by
> >     biosbits
> >   acpi/tests/bits: disable acpi PSS tests that are failing in biosbits
> >   acpi/tests/bits: add smilatency test suite from bits in order to
> >     disable it
> >   acpi/tests/bits: disable smilatency test since it does not pass
> >     everytime
> >   acpi/tests/bits: add biosbits config file for running bios tests
> >   acpi/tests/bits: add acpi and smbios python tests that uses biosbits
> >   acpi/tests/bits: add acpi bits qtest directory in meson for running
> >     tests
> >   acpi/tests/bits: add README file for bits qtests
> >   MAINTAINERS: add myself as the maintainer for acpi biosbits qtests
> >
> >  MAINTAINERS                                   |    5 +
> >  tests/qtest/acpi-bits/README                  |  168 ++
> >  tests/qtest/acpi-bits/acpi-bits-test-venv.sh  |   59 +
> >  tests/qtest/acpi-bits/acpi-bits-test.py       |  327 +++
> >  .../qtest/acpi-bits/bits-config/bits-cfg.txt  |   18 +
> >  tests/qtest/acpi-bits/bits-config/meson.build |   11 +
> >  tests/qtest/acpi-bits/bits-tests/meson.build  |   11 +
> >  tests/qtest/acpi-bits/bits-tests/smbios.py    | 2430 +++++++++++++++++
> >  .../qtest/acpi-bits/bits-tests/smilatency.py  |  103 +
> >  tests/qtest/acpi-bits/bits-tests/testacpi.py  |  283 ++
> >  tests/qtest/acpi-bits/bits-tests/testcpuid.py |   83 +
> >  tests/qtest/acpi-bits/meson.build             |   39 +
> >  .../acpi-bits/prebuilt/bits-2095-grub.tar.gz  |  Bin 0 -> 41416278 bytes
> >  tests/qtest/acpi-bits/prebuilt/bits-2095.zip  |  Bin 0 -> 31922898 bytes
> >  tests/qtest/acpi-bits/prebuilt/meson.build    |   11 +
> >  tests/qtest/acpi-bits/requirements.txt        |    1 +
> >  tests/qtest/meson.build                       |    7 +-
> >  17 files changed, 3555 insertions(+), 1 deletion(-)
> >  create mode 100644 tests/qtest/acpi-bits/README
> >  create mode 100644 tests/qtest/acpi-bits/acpi-bits-test-venv.sh
> >  create mode 100644 tests/qtest/acpi-bits/acpi-bits-test.py
> >  create mode 100644 tests/qtest/acpi-bits/bits-config/bits-cfg.txt
> >  create mode 100644 tests/qtest/acpi-bits/bits-config/meson.build
> >  create mode 100644 tests/qtest/acpi-bits/bits-tests/meson.build
> >  create mode 100644 tests/qtest/acpi-bits/bits-tests/smbios.py
> >  create mode 100644 tests/qtest/acpi-bits/bits-tests/smilatency.py
> >  create mode 100644 tests/qtest/acpi-bits/bits-tests/testacpi.py
> >  create mode 100644 tests/qtest/acpi-bits/bits-tests/testcpuid.py
> >  create mode 100644 tests/qtest/acpi-bits/meson.build
> >  create mode 100644 tests/qtest/acpi-bits/prebuilt/bits-2095-grub.tar.gz
> >  create mode 100644 tests/qtest/acpi-bits/prebuilt/bits-2095.zip
> >  create mode 100644 tests/qtest/acpi-bits/prebuilt/meson.build
> >  create mode 100644 tests/qtest/acpi-bits/requirements.txt
> >
> > --
> > 2.25.1
> >