Re: [Qemu-devel] [RFC] Future goals for autotest and virtualization tests

[Anthony Liguori]

On 03/08/2012 09:57 AM, Ademar Reis wrote:
> On Thu, Mar 08, 2012 at 04:46:09PM +0100, Kevin Wolf wrote:
> > Am 08.03.2012 15:56, schrieb Anthony Liguori:
> > > > I particularly agreed with basically everything you said on that
> > > > discussion regarding test simplification (I had just joined the
> > > > team back then). To me, autotest has been focusing on QE-level,
> > > > leaving the developer-level test requirements out. Now we're
> > > > attacking this new front, and a lot of the requirements are
> > > > indeed from that discussion.
> > >
> > > If you want to talk about this in terms of "requirements", my requirement is for
> > > "developer-level" tests to live in qemu.git and be integrated into make check.
> >
> > Actually, I think make check should be something that runs _really_
> > quickly. We'll probably want another make check-full or something that
> > runs more extensive tests.
>
> I like make check because it's part of make, but I would suggest:
>
> $ test-runner<dir>  |<file>  ...
>
> This would allow use-cases such as:
>
> [qemu]$ test-runner tests.d/block/
>    (run all block tests)

make check-block

> [qemu]$ test-runner tests.d/core/whatever-i-want-to-test.sh
>    (run one individual test)

./tests/core/whatever-i-want-to-test

> [qemu]$ test-runner tests.d/
>    (run all tests)

make check

> [qemu]$ test-runner tests.d/devices/*pci*
>    (you get the idea)

make check-pci

> And, of course:
> [qemu]$ test-runner --remote=autotest.qemu.org tests.d/block

I don't understand what this would do.

Are you proposing test-runner as a replacement for gtest?

Regards,

Anthony Liguori

> (a test path/name is its ID to both the local test runner and the
> remote server)
>
> Cheers,
>    - Ademar
>
> > That's the reason why the patch I proposed earlier today runs only a few
> > qemu-iotests cases during make check. The whole thing would already take
> > too long for everyone to use it. If people run some quick sanity tests
> > for all of qemu plus the full tests in the subsystem in which they're
> > making changes, I think that would be the best we can realistically expect.
> >
> > Kevin