Re: [Qemu-devel] [PATCH] Acceptance tests: host arch to target arch name mapping

[Cleber Rosa]

On 10/17/18 2:40 PM, Peter Maydell wrote:
> On 17 October 2018 at 18:38, Cleber Rosa <address@hidden> wrote:
>>
>>
>> On 10/17/18 12:29 PM, Eduardo Habkost wrote:
>>> On Wed, Oct 17, 2018 at 01:34:41PM +0100, Peter Maydell wrote:
>>>> So, why does the test code need to care? It's not clear
>>>> from the patch... My expectation would be that you'd
>>>> just test all the testable target architectures,
>>>> regardless of what the host architecture is.
>>>
>>> I tend to agree.  Maybe the right solution is to get rid of the
>>> os.uname().  I think the default should be testing all QEMU
>>> binaries that were built, and the host architecture shouldn't
>>> matter.
> 
> Yes, looking at os.uname() also seems like an odd thing
> for the tests to be doing here. The test framework
> should be as far as possible host-architecture agnostic.
> (For some of the KVM cases there probably is a need to
> care, but those are exceptions, not the rule.)
> 
>> I'm in favor of exercising all built targets, but that seems to me to be
>> on another layer, above the test themselves. This change is about the
>> behavior of a test when not told about the target arch (and thus binary)
>> it should use.
> 
> At that level, I think the right answer is "tell the user
> they need to specify the qemu executable they are trying to test".
> In particular, there is no guarantee that the user has actually
> built the executable for the target that corresponds to the
> host, so it doesn't work to try to default to that anyway.
> 

I think your "right answer" is the ideal, but I'm afraid it's not
practical.  For instance, look at how a qemu-iotest is run:

$ ./check 001
QEMU          --
"/tmp/qemu-build/tests/qemu-iotests/../../ppc64-softmmu/qemu-system-ppc64"
-nodefaults -machine accel=qtest
...

IIRC, you're suggesting that check SHOULD abort and require a QEMU
binary.  The command line would then become something like:

$ ./check --qemu-bin=../../ppc64-softmmut/qemu-system-ppc64 001

The rules that "check" uses for finding a "default" QEMU binary are a
lot harder to describe (are "smarter") than the ones we currently have
on the acceptance tests.  IMO some compromises like these are necessary,
or else, tests may be run much much less than they could and should.

When it comes to trying to run the acceptance tests without an existing
QEMU binary built, the current behavior is to CANCEL the tests.

Regards,
- Cleber.

> thanks
> -- PMM
>