[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [FYI] {maint} tests: report useful system information from 'test-sui
From: |
Stefano Lattarini |
Subject: |
Re: [FYI] {maint} tests: report useful system information from 'test-suite.log' |
Date: |
Thu, 22 Dec 2011 15:58:13 +0100 |
User-agent: |
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.24) Gecko/20111114 Icedove/3.1.16 |
Hi Eric.
On 12/22/2011 03:41 PM, Eric Blake wrote:
> On 12/22/2011 07:30 AM, Stefano Lattarini wrote:
>> * tests/get-sysconf.test: New test, gathering system information
>> and then always terminating with a SKIP, so that its output gets
>> copied in `test-suite.log'.
>> * tests/Makefile.am (TESTS): Add it.
>
> I like the idea. But I'm afraid it might introduce a few bug reports on
> its own ("I ran the testsuite, and it skipped 1 test - how do I fix
> things to not skip that test?").
>
Well, time will tell :-) But it's worth noting that at least one test in
the automake testsuite is bound to be skipped anyway on almost every system:
some tests require libtool macros *in /usr/local/share/aclocal* (if one
has configured automake with the default prefix), some requires Vala, some
the UPC compiler, some the Sun java interpreter ... (you've got the gist I
guess), and nobody has ever complained so far about this situation. So I
guess we're pretty safe.
> I also worry that since config.log can be long,
>
Luckily, the automake's one isn't (even in the branches that have started
looking for compilers at configure time).
> it might make the user's
> email long enough to trip up list posting limitations, and not all users
> know how to compress things to get around that.
>
If we come to that, we should raise the list's post size limitation IMO.
> And even if they do
> compress it, it's that much harder to read the testsuite failure report.
>
> But I don't have any ideas for how to do better, so you might as well go
> with it.
>
Yeah, IMHO it's a definite improvement over the previous situation. And
if we find a better solution in the future, well, we can simply implement
it, right? :-)
>> +# Dummy test case, aimed at reporting useful system information in the
>> +# final `test-suite.log'. This way, if a user experiencing a failure in
>> +# the Automake test suite only sends us the `test-suite.log' file upon,
>> +# we won't have to ask him for more information to start analyzing the
>> +# failure (that had happened too many times in the past!).
>
> grammar:
>
> This way, if a user experiences a failure in the Automake test suite,
> then only sends us the `test-suite.log', we won't have to...
>
Will fix in a follow-up (probably this evening).
>> +
>> +if test $st -eq 0; then
>> + # This test SKIPs, so that all the information is has gathered and
>
> s/is has //
>
Ditto.
>> + # printed will get unconditionally copied into the `test-suite.log'
>> + # file.
>> + Exit 77
>> +fi
>> +
>> +# Some unexpected error occurred; this must be reported as an hard error
>> +# by the testsuite driver.
>> +Exit 99
>
Thanks,
Stefano