[PATCH 2/?] Do not SKIP a test on a command failing with `77' exit status (was: [PATCH] New requirement "cc" for tests using a C compiler)

[Stefano Lattarini]

At Tuesday 22 June 2010, Stefano Lattarini wrote:
> > Next step will be to run the whole automake testsuite with
> > `CC=no', to see which tests need a C compiler without explicitly
> > requiring it.
> 
> Which obviously doesn't help, because a configure script containing
>  the expansion of AC_PROG_CC exits with status `77' if it can't
>  find a working C compiler, so that the test script is skipped. 
>  Hmm... there is a simple way to disable this globally (e.g.
>  config.site, environment variable, ...), forcing configure to exit
>  with status `1' in such a situation?
> 
Never mind, I wrote a patch to make test scripts using `set -e' 
resulting as a FAIL rather than as a SKIP if a command in them exists 
with status 77. 

Obviously, a test can still be skipped by using explicit calls to 
`Exit 77', as:
  # SKIP if we miss the cscope program
  cscope --version || Exit 77
or even:
  # SKIP the test if configure suggests to do so, otherwise FAIL it if
  # configure exit with status != 0
  ./configure || Exit $?

I think that forcing ourselves to be explicit about test skips could 
help us to better find out hidden dependencies, and to avoid false 
negatives (in the form of unexpected skips).  So, even if I had 
originally considered this patch just as a temporary hack (used to
see which tests need a C compiler without explicitly requiring it),
I now *strongly* believe that it should be applied upstream.  WDYT?

Regards,
    Stefano

From 5444ed936ccf08cc1979ee404feb81e9037382ad Mon Sep 17 00:00:00 2001
From: Stefano Lattarini <address@hidden>
Date: Wed, 23 Jun 2010 00:30:46 +0200
Subject: [PATCH 2/4] Do not SKIP a test on a command failing with `77' exit 
status.

When, in a test script using the `errexit' (`-e') shell flag,
a command failed with status `77' the global test outcome was
considered a SKIP, bacause the a value of `77' for `$?' was
passed to the exit trap.  This could happen in practice, as an
autoconf-generated configure script exits with status `77' if
it fails to find e.g. a required compiler.  This patch modify
the exit trap, so that only an explicit `Exit 77' makes the
test outcome be considered a SKIP.

* tests/defs.in ($am_test_skipped): New variable, initialized
to "no".
(Exit): Set $am_test_skipped to "yes" if passed an exit status
of 77.
(trap '...' 0): Reset exit status to `1' if it is = `77', but
$am_test_skipped is not set to "yes".
* tests/upc.test: Skip it explicitly if ./configure exit with
status 77.
* tests/upc3.test: Likewise.
---
 ChangeLog       |   21 +++++++++++++++++++++
 tests/defs.in   |    9 +++++++++
 tests/upc.test  |    4 +++-
 tests/upc3.test |    4 +++-
 4 files changed, 36 insertions(+), 2 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index 933047d..369e27e 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,4 +1,25 @@
 2010-06-22  Stefano Lattarini  <address@hidden>
+
+       Do not SKIP a test on a command failing with `77' exit status.
+       When, in a test script using the `errexit' (`-e') shell flag,
+       a command failed with status `77' the global test outcome was
+       considered a SKIP, bacause the a value of `77' for `$?' was
+       passed to the exit trap.  This could happen in practice, as an
+       autoconf-generated configure script exits with status `77' if
+       it fails to find e.g. a required compiler.  This patch modify
+       the exit trap, so that only an explicit `Exit 77' makes the
+       test outcome be considered a SKIP.
+       * tests/defs.in ($am_test_skipped): New variable, initialized
+       to "no".
+       (Exit): Set $am_test_skipped to "yes" if passed an exit status
+       of 77.
+       (trap '...' 0): Reset exit status to `1' if it is = `77', but
+       $am_test_skipped is not set to "yes".
+       * tests/upc.test: Skip it explicitly if ./configure exit with
+       status 77.
+       * tests/upc3.test: Likewise.
+
+2010-06-22  Stefano Lattarini  <address@hidden>
            Ralf Wildenhues  <address@hidden>
 
        New requirement "cc" for tests using a C compiler.
diff --git a/tests/defs.in b/tests/defs.in
index 1d6af76..3fef7b1 100644
--- a/tests/defs.in
+++ b/tests/defs.in
@@ -254,6 +254,11 @@ case "$srcdir" in
     ;;
 esac
 
+# This is to ensure that a test script does not result as a SKIP
+# just because a command in it happens to exit with status 77 when
+# the `errexit' (aka `set -e') shell flag is active.
+am_test_skipped=no
+
 # We use a trap below for cleanup.  This requires us to go through
 # hoops to get the right exit status transported through the signal.
 # So use `Exit STATUS' instead of `exit STATUS' inside of the tests.
@@ -262,6 +267,7 @@ esac
 Exit ()
 {
   set +e
+  test 77 = $1 && am_test_skipped=yes
   (exit $1)
   exit $1
 }
@@ -278,6 +284,9 @@ address@hidden@
 if test "$sh_errexit_works" = yes; then
   trap 'exit_status=$?
     set +e
+    if test $exit_status -eq 77 && test x"$am_test_skipped" != x"yes"; then
+      exit_status=1
+    fi
     cd "$curdir"
     case $exit_status,$keep_testdirs in
     0,)
diff --git a/tests/upc.test b/tests/upc.test
index 3c55ccb..725166c 100755
--- a/tests/upc.test
+++ b/tests/upc.test
@@ -45,5 +45,7 @@ $ACLOCAL
 $AUTOMAKE
 $AUTOCONF
 
-./configure
+# Skip the test if configure suggests so.
+./configure || Exit $?
+
 $MAKE distcheck
diff --git a/tests/upc3.test b/tests/upc3.test
index f575b9a..870c66e 100755
--- a/tests/upc3.test
+++ b/tests/upc3.test
@@ -60,5 +60,7 @@ $ACLOCAL
 $AUTOMAKE
 $AUTOCONF
 
-./configure
+# Skip the test if configure suggests so.
+./configure || Exit $?
+
 $MAKE distcheck
-- 
1.6.5