[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[FYI 2/6] {testsuite-work} testsuite: more granular count of test result
From: |
Stefano Lattarini |
Subject: |
[FYI 2/6] {testsuite-work} testsuite: more granular count of test results in our TAP library |
Date: |
Sun, 14 Aug 2011 16:37:54 +0200 |
* tests/tap-functions.sh ($tap_bad_count_): Removed, superseded
by ...
($tap_fail_count_, $tap_xpass_count_): ... these new variables,
which keep more granular counts.
($tap_pass_count_: New variable.
* tests/defs (exit trap): Adjust and simplify accordingly.
---
ChangeLog | 10 ++++++++++
tests/defs | 5 +----
tests/tap-functions.sh | 15 ++++++++-------
3 files changed, 19 insertions(+), 11 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index 1d4ee09..cd30f9f 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,15 @@
2011-08-13 Stefano Lattarini <address@hidden>
+ testsuite: more granular count of test results in our TAP library
+ * tests/tap-functions.sh ($tap_bad_count_): Removed, superseded
+ by ...
+ ($tap_fail_count_, $tap_xpass_count_): ... these new variables,
+ which keep more granular counts.
+ ($tap_pass_count_: New variable.
+ * tests/defs (exit trap): Adjust and simplify accordingly.
+
+2011-08-13 Stefano Lattarini <address@hidden>
+
testsuite: fixlets and improvements in two long TAP-based tests
* tests/depmod.tap: Clean up the subdirectories used by tests that
passed, to avoid ending up with a too big test directory. This is
diff --git a/tests/defs b/tests/defs
index 047e22f..3bd378e 100644
--- a/tests/defs
+++ b/tests/defs
@@ -871,10 +871,7 @@ if test "$sh_errexit_works" = yes; then
if test $have_tap_plan_ != yes; then
plan_ "now"
fi
- test $exit_status -eq 0 \
- && test $tap_xfail_count_ -eq 0 \
- && test $tap_skip_count_ -eq 0 \
- && test $tap_bad_count_ -eq 0 \
+ test $exit_status -eq 0 && test $tap_pass_count_ -eq $tap_count_ \
|| keep_testdirs=yes
else
# This is to ensure that a test script does give a SKIP outcome just
diff --git a/tests/tap-functions.sh b/tests/tap-functions.sh
index 603b53c..ea88887 100644
--- a/tests/tap-functions.sh
+++ b/tests/tap-functions.sh
@@ -22,14 +22,14 @@
# from within a subshell, unless explicitly noted otherwise.
#
-# The count of the TAP test results seen so far.
+# The counts of the TAP test results seen so far: total count and
+# per-result counts.
tap_count_=0
-# The count of skipped tests.
+tap_pass_count_=0
tap_skip_count_=0
-# The count of tests that experienced an expected failure.
+tap_fail_count_=0
tap_xfail_count_=0
-# The count of tests with unexpected outcomes (i.e., failed and xpassed).
-tap_bad_count_=0
+tap_xpass_count_=0
# The first "test -n" tries to avoid extra forks when possible.
if test -n "${ZSH_VERSION}${BASH_VERSION}" \
@@ -132,9 +132,10 @@ result_ ()
esac
incr_ tap_count_
case $tap_result_,$tap_directive_ in
- ok,) ;; # Passed.
+ ok,) incr_ tap_pass_count_;; # Passed.
not\ ok,TODO) incr_ tap_xfail_count_;; # Expected failure.
- not\ ok,*|ok,TODO) incr_ tap_bad_count_ ;; # Failed or xpassed.
+ not\ ok,*) incr_ tap_fail_count_ ;; # Failed.
+ ok,TODO) incr_ tap_xpass_count_ ;; # Unexpected pass.
ok,SKIP) incr_ tap_skip_count_ ;; # Skipped.
*) bailout_ "internal error in 'result_'";; # Can't happen.
esac
--
1.7.2.3
- [FYI 0/6] {testsuite-work} More testsuite patches, Stefano Lattarini, 2011/08/14
- [FYI 1/6] {testsuite-work} testsuite: fixlets and improvements in two long TAP-based tests, Stefano Lattarini, 2011/08/14
- [FYI 2/6] {testsuite-work} testsuite: more granular count of test results in our TAP library,
Stefano Lattarini <=
- [FYI 3/6] {testsuite-work} testsuite: keep more debugging info around in 'add-missing.tap', Stefano Lattarini, 2011/08/14
- [FYI 4/6] {testsuite-work} testsuite: avoid confusing SKIP messages for errored tests, Stefano Lattarini, 2011/08/14
- [FYI 5/6] {testsuite-work} maintcheck: look for TAP-based tests without TAP plan, Stefano Lattarini, 2011/08/14
- [FYI 6/6] {testsuite-work} self tests: some minor refactoring, Stefano Lattarini, 2011/08/14