Re: [Qemu-devel] [PATCH v4 0/6] iotests: Allow out-of-tree run

[Markus Armbruster]

Max Reitz <address@hidden> writes:

> On 07.06.2014 23:21, Max Reitz wrote:
>> On 24.05.2014 23:24, Max Reitz wrote:
>>> This series enables qemu-iotests to be run in a build tree outside of
>>> the source tree. It also makes the tests use the command for invoking
>>> the Python interpreter specified through configure instead of always
>>> using "/usr/bin/env python".
>>
>> Ping; I do understand that this series is not urgent, but since I
>> realized out-of-tree builds to be probably superior, I personally
>> base all my own patches on this series, as I don't want to fiddle
>> around with the iotests. Therefore, I'd be glad if someone would
>> review the remaining patches so it can be merged soon. :-)
>
> Ping again. Because this is just convenient for development, I don't
> need it in any specific release, though.

I haven't found the time for a proper review, and I can't promise one
right now, so I should probably keep my mouth where my money is, but
here goes anyway: unless running tests is utterly trivial, tests will
not be run, and avoidable mistakes happen.

Case in point: I spent a non-trivial chunk of time yesterday to debug
three regressions clearly visible in iotests.  I did not scold the
people involved in getting the regressions committed for not running
these tests, because I feel strongly I can't demand tests to be run that
require instructions more complex than "make WHATEVER".

I don't think this is "just convenient for development".  I'd say it's a
must-have.