qemu-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Qemu-devel] [PATCH 0/4] add "make check"


From: Lucas Meneghel Rodrigues
Subject: Re: [Qemu-devel] [PATCH 0/4] add "make check"
Date: Tue, 25 Oct 2011 14:30:09 -0200
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:7.0.1) Gecko/20110930 Thunderbird/7.0.1

On 10/25/2011 01:03 PM, Eduardo Habkost wrote:
On Tue, Oct 25, 2011 at 03:27:35PM +0200, Gerd Hoffmann wrote:
[...]
A while ago I played with some simple IDE tests. It basically was a
small x86 kernel with an empty image that sends IDE commands and prints
some results, and a script that invokes the guest and checks whether the
test has passed or failed.

That reminds me that I've started toying with running tests inside a
guest too.  Stopped working on it a while back due to other priorities.
  Attached what I have so far.

So at first I started with my own multiboot kernel and copied over some
parts of kvm-unittest's libc. Clearly not the best idea once it's more
than a couple of lines, so at some point I took the code and integrated
with my real kvm-unittests repository.

Now I don't have to duplicate code any more, but at the same time
there's no chance that a 'make check' in an upstream qemu tree could run
this. Tests for other devices will have exactly the same problem.

Any suggestions on how to go forward with this kind of tests? Should
this go into qemu or into kvm-unittests? Or should kvm-unittests be
merged into the qemu tree? Or is the approach completely wrong?

I think we should have some framework to run tests inside the guest in
the qemu source tree.  I'm not sure kvm-unittests is the right tool for
the job though.  It is quite low-level and mainly targets the kvm bits
inside the linux kernel.  Testing -- for example -- usb device emulation
would pretty much require writing a usb stack for kvm-unitests ...

We have a framework to run tests inside a fully-installed guest, that's
KVM-Autotest. But maybe it's too much for the kind of tests you need, I
don't know. There are different "levels" of testing, with different
reequirements, and we need to have good tools for all levels.

Just trying to enumerate the kind of tests somebody may need:

A) Simple unit tests for internal qemu C functions
    - 'make check' can run them, using either libcheck or gtest.
B) Functional tests that tests actual virtualization/emulation, but only
    of some specific subsystems, not using a fully-featured qemu process.
    - We don't have anything like that, today, right? I am not sure we
      need it.
C) Functional tests that just need to run a small binary with no OS
    installed in the guest, but running a fully-feature qemu process.
    - The tests in the 'tests' directory do this, right? kvm-unittests
      does this, right?
D) Functional tests that need a minimal OS installed, with, e.g., at
    least a Linux kernel and a shell.
    - This is what Gerd's patch below does, right? Also, KVM-Autotest can
      be used for this.

Yes, it can. There's instrumentation to get ssh sessions, serial sessions, execute monitor commands... Only need to specify the path to a pre-installed minimal guest, skip install tests and have the bare minimal subset of tests needed. If only a small subset of tests are run with a minimal image, a kvm autotest job is quick to run and suitable for development use. I've looked at Gerd's initial patch to add an initramfs for such lightweight testing, and the init ramdisk generation could be implemented in autotest.

However...

Problem today is that KVM autotest is considered to be hard to approach and integrate with qemu developer's workflow. I won't argue it's not, so I'm admitting failure here. We've been juggling to keep it stable for the regular, full blown QE case, with expanding test and targets coverage (see libvirt test), and yet trying to make it better documented and sufficiently easy to approach to do simpler ad hoc testing. I wish it was easier to accomplish the later, though.

So, an in-tree solution for D) like the one Gerd started to work on seems better, as it won't generate frustration from folks having to learn kvm autotest, which will ultimately lead to them ignoring the effort.

E) Functional tests that need a full OS installed and configured.
    - Today we use KVM-Autotest for this.


Does the above model look correct/complete, or is there some case I
missed?



cheers,
   Gerd

 From 096f68ea08c3c4baf1bbdc549b257a67ecc87e25 Mon Sep 17 00:00:00 2001
From: Gerd Hoffmann<address@hidden>
Date: Tue, 13 Sep 2011 17:38:37 +0200
Subject: [PATCH] initramfs test framework

Signed-off-by: Gerd Hoffmann<address@hidden>
---
  initramfs/.gitignore         |    3 +
  initramfs/10-qemu-udev.rules |    5 +
  initramfs/Makefile           |   36 +++++++
  initramfs/README             |   44 ++++++++
  initramfs/init.c             |  225 ++++++++++++++++++++++++++++++++++++++++++
  initramfs/initramfs-boot     |   32 ++++++
  initramfs/initramfs-create   |  111 +++++++++++++++++++++
  initramfs/test-ehci          |    3 +
  initramfs/test-ehci.good     |    8 ++
  initramfs/test-hello.c       |    7 ++
  initramfs/test-hello.good    |    1 +
  initramfs/test-uhci          |    3 +
  initramfs/test-uhci.good     |    3 +
  13 files changed, 481 insertions(+), 0 deletions(-)
  create mode 100644 initramfs/.gitignore
  create mode 100644 initramfs/10-qemu-udev.rules
  create mode 100644 initramfs/Makefile
  create mode 100644 initramfs/README
  create mode 100644 initramfs/init.c
  create mode 100755 initramfs/initramfs-boot
  create mode 100755 initramfs/initramfs-create
  create mode 100755 initramfs/test-ehci
  create mode 100644 initramfs/test-ehci.good
  create mode 100644 initramfs/test-hello.c
  create mode 100644 initramfs/test-hello.good
  create mode 100755 initramfs/test-uhci
  create mode 100644 initramfs/test-uhci.good






reply via email to

[Prev in Thread] Current Thread [Next in Thread]