Re: [Qemu-devel] [RFC] Future goals for autotest and virtualization tests

[Kevin Wolf]

Am 09.03.2012 00:51, schrieb Ademar Reis:
> On Thu, Mar 08, 2012 at 05:21:44PM -0600, Anthony Liguori wrote:
>>> Plus it's not unconditional: the test runner will report tests
>>> SKIPPED if a dependency is not present.
>>
>> But then the tests aren't run so if most developers didn't have it
>> installed, and most tests were written with it, most developers
>> wouldn't be running most tests which defeats the purpose, no?
> 
> Part of a TDD approach is to have build and test bots frequently
> running tests on multiple platforms with different
> configurations.
> 
> You can't expect developers to run all tests all the time.

I think this is one of the most important points: Not all developers
must run all the tests all the time.

Actually, Anthony agreed with me when I said that developers should run
some sanity tests for all of qemu and maybe a few more tests for the
subsystems they're touching. I agree that it would be bad to have a
autotest dependency for those basic tests that everyone should run and
that should take a few minutes at most.

For the rest of test cases, however, not everyone needs to run them and
I think an external dependency (that is reasonably easy to satisfy) is
not a problem there. A test bot would be great, but even if people just
run them occasionally by hand, that would already detect bugs that are
currently left in the code for months. If maintainers do it before
pushing code into master, you'll even catch everything before it goes
into master. This is as good as it gets.

The important thing is that tests exist in the first place, not who runs
them.

Kevin