[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH] Fix exit status of signal handlers in shell scripts
From: |
Jim Meyering |
Subject: |
Re: [PATCH] Fix exit status of signal handlers in shell scripts |
Date: |
Mon, 01 Feb 2010 08:58:19 +0100 |
Bruno Haible wrote:
> Jim Meyering wrote:
>> Imagine that the first 10 tests pass, then each of the remaining ones is
>> killed via e.g., SIGHUP. ...
>> a naive search for "FAIL:" in the build output would find nothing.
>
> Yes, and it should be this way, IMO. Each time a user sees a "FAIL:", he
> should be encouraged to investigate.
>
> Whereas in the gettext test suite, often when I sent SIGINTs, I saw some
> tests fail without explanation. (This was due to a missing 'exit' statement
> in the trap handler, but it would be the same if there was an 'exit 1' in
> the trap handler.) I guessed that the FAIL report was due to the SIGINT and
> did not investigate. But I don't think this attitude should be encouraged.
>
> Similarly, when I get reports from Nelson Beebe with lots of failing tests,
> I don't want to spend time on fake failures that were due to, maybe, a
> shutdown of his virtual machine or something like this.
>
>> The final result would be highly misleading:
>>
>> ========================
>> All 10 tests passed
>> (300 tests were not run)
>> ========================
>
> But before this final result, you would see 300 times
>
> Skipping test: caught fatal signal
> SKIP: test-foo1
> Skipping test: caught fatal signal
> SKIP: test-foo2
> Skipping test: caught fatal signal
> SKIP: test-bar
> ...
>
> That should be enough of an explanation, no? And it will tell us that there's
> no gnulib bug to investigate.
I'm sure it's enough if you are alert and watching all
of the output go by, but that is not reliable.
More often, I redirect to a file and search for
traces of unexpected behavior (usually failure) by running
"grep FAIL:" on the output.
Letting automake report "all 10 tests passed" is misleading.
I want my tests to FAIL whenever something unexpected happens,
be it user interrupt via control-C or a SIGHUP sent by some
other application. If a user interrupts "make check", there
is little risk s/he will report that as a legitimate test failure.
However, if something is misbehaving and killing my shells when
it should not, I don't want to overlook that because some test
framework decided to classify that test as merely "SKIPPED".
[Prev in Thread] |
Current Thread |
[Next in Thread] |
- Re: [PATCH] Fix exit status of signal handlers in shell scripts,
Jim Meyering <=