Re: [Qemu-devel] [PATCH v2 5/5] tests: add functional test validating ipv4/ipv6 address flag handling

[Eric Blake]

On 05/22/2017 11:56 AM, Daniel P. Berrange wrote:
> On Mon, May 22, 2017 at 11:00:26AM -0500, Eric Blake wrote:
>> On 05/19/2017 01:03 PM, Daniel P. Berrange wrote:
>>> The semantics around handling ipv4=on|off & ipv6=on|off are quite
>>> subtle to understand in combination with the various hostname addresses
>>> and backend types. Introduce a massive test matrix that launches QEMU
>>> and validates the ability to connect a client on each protocol as
>>> appropriate.
>>>
>>> The test requires that the host has ability to bind to both :: and
>>> 0.0.0.0, on port 9000. If either protocol is not available, or if
>>> something is already listening on that port the test will skip.
>>>

>>
>> 'make check' passed for me with your patches (on a system with both IPv4
>> and IPv6 support), so:
>>
>> Tested-by: Eric Blake <address@hidden>
>>

I take that back. While the sequential version passed, 'make -j3 check'
failed:

ERROR:tests/test-sockets-proto.c:774:test_listen: assertion failed:
(data->error)
GTester: last random seed: R02Sb234b038cf2cb256668c3f8d68b73cb6
**
ERROR:tests/test-sockets-proto.c:805:test_listen: assertion failed:
(child != 0)
GTester: last random seed: R02S9da269b6008df86a92d3fa931b311f54
**
ERROR:tests/test-sockets-proto.c:774:test_listen: assertion failed:
(data->error)
GTester: last random seed: R02S40fd0cddf35acf3acdde9c1d0bc5430c

Probably because it raced between two separate qtest programs (in my
case, check-qtest-x86_64 ran in parallel with check-qtest-ppc64), at
which point there was competition for port 9000 that survived past the
initial check but not for the complete test run.

Can you make the test a bit more robust by finding a free port during
initial setup for all the tests to hammer on, rather than hard-coding 9000?

>>> +int main(int argc, char **argv)
>>> +{
>>> +    int ret;
>>> +    gsize i;
>>> +
>>> +    if (check_protocol_support() < 0) {
>>> +        return 0; /* Skip test if we can't bind */
>>
>> We don't have a magic number for skipped tests?  Many projects use exit
>> status 77 (rather than 0) to delineate a test that did not fail, but
>> whose skip results cannot be used as conclusive evidence of passing.  At
>> any rate, you've answered my question earlier - you do behave sanely if
>> you cannot test both IPv4 and IPv6 on a given host.
> 
> I thought about using status 77, but that doesn't seem to be done in
> QEMU unit tests, so I stuck with 0.

Yeah, cleaning that up is a separate series, if ever.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

signature.asc
Description: OpenPGP digital signature