Re: [PATCH] Fix exit status of signal handlers in shell scripts

[Dmitry V. Levin]

On Mon, Feb 01, 2010 at 08:58:19AM +0100, Jim Meyering wrote:
> Bruno Haible wrote:
> > Jim Meyering wrote:
> >> Imagine that the first 10 tests pass, then each of the remaining ones is
> >> killed via e.g., SIGHUP. ...
> >> a naive search for "FAIL:" in the build output would find nothing.
> >
> > Yes, and it should be this way, IMO. Each time a user sees a "FAIL:", he
> > should be encouraged to investigate.
> >
> > Whereas in the gettext test suite, often when I sent SIGINTs, I saw some
> > tests fail without explanation. (This was due to a missing 'exit' statement
> > in the trap handler, but it would be the same if there was an 'exit 1' in
> > the trap handler.) I guessed that the FAIL report was due to the SIGINT and
> > did not investigate. But I don't think this attitude should be encouraged.
> >
> > Similarly, when I get reports from Nelson Beebe with lots of failing tests,
> > I don't want to spend time on fake failures that were due to, maybe, a
> > shutdown of his virtual machine or something like this.
> >
> >> The final result would be highly misleading:
> >>
> >>     ========================
> >>     All 10 tests passed
> >>     (300 tests were not run)
> >>     ========================
> >
> > But before this final result, you would see 300 times
> >
> >   Skipping test: caught fatal signal
> >   SKIP: test-foo1
> >   Skipping test: caught fatal signal
> >   SKIP: test-foo2
> >   Skipping test: caught fatal signal
> >   SKIP: test-bar
> >   ...
> >
> > That should be enough of an explanation, no? And it will tell us that 
> > there's
> > no gnulib bug to investigate.
> 
> I'm sure it's enough if you are alert and watching all
> of the output go by, but that is not reliable.
> 
> More often, I redirect to a file and search for
> traces of unexpected behavior (usually failure) by running
> "grep FAIL:" on the output.
> 
> Letting automake report "all 10 tests passed" is misleading.
> 
> I want my tests to FAIL whenever something unexpected happens,
> be it user interrupt via control-C or a SIGHUP sent by some
> other application.  If a user interrupts "make check", there
> is little risk s/he will report that as a legitimate test failure.
> However, if something is misbehaving and killing my shells when
> it should not, I don't want to overlook that because some test
> framework decided to classify that test as merely "SKIPPED".

I just want to remind you that the undefined behaviour still hasn't been
fixed.  Please make a decision what kind of fix suits you better.


-- 
ldv

pgp2IS3Y48yLS.pgp
Description: PGP signature