Re: [Qemu-devel] What tests should "make check-block" run?

[Markus Armbruster]

Stefan Hajnoczi <address@hidden> writes:

> On Thu, Aug 21, 2014 at 02:16:36PM +0100, Peter Maydell wrote:
>> On 21 August 2014 14:12, Stefan Hajnoczi <address@hidden> wrote:
>> > On Thu, Aug 21, 2014 at 02:27:00PM +0200, Markus Armbruster wrote:
>> >> Should we have a variant of "make check-block" for testing other
>> >> (format, protocol) combinations?
>> >
>> > I don't think variants are useful.  If you need control, use ./check.
>> 
>> That seems pretty undiscoverable to me. I know about 'make check',
>> and 'make check-help' tells me about 'make check-block', but how
>> do I find out about 'check' ? I just had to bounce through the makefile
>> and a wrapper script to even figure out which directory it lives in,
>> and there's no help text or usage comments in it...
>
> http://qemu-project.org/Documentation/QemuIoTests

I'm afraid this needs updating for Max's recent "iotests: Allow
out-of-tree run" series.  Max?

> Any suggestions for making it more discoverable?

How do you discover tests?

Developers familiar with GNU-ish make target conventions will probably
try "make check".  If it works (and it does in QEMU), then that's that.

People who actually read build instructions exist.  For those people,
the instructions should describe how to run tests.  Try 'git-grep "make
check" and weep.

"make check" won't cut it for the block layer, you need "make
check-block" for a quick test, and different, still more involved
incantations for a thorough test.

"make help" could lead people to idiosyncratic targets like check-block.
No go, it's "make check-help".  Nice reminder if you know it already,
but it's not discoverable.

I think we could do a few things:

* Have make check print "More tests are available, run 'make check-help'
  to list them" at the very end.

* Either have a make target for thorough block tests, or have check-help
  point to instructions.

* "make help" would be nice to have.

* Have a file INSTALL describing how to build and install this beast.
  Autoconf comes with one fit for autoconfiscated projects.  QEMU isn't,
  but it's vaguely similar, so autoconf's INSTALL could serve as a
  starting point.