[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Automake-commit] [SCM] GNU Automake branch, test-protocols, updated. v1
|
From: |
Stefano Lattarini |
|
Subject: |
[Automake-commit] [SCM] GNU Automake branch, test-protocols, updated. v1.11-960-gbc5ba17 |
|
Date: |
Fri, 05 Aug 2011 08:31:57 +0000 |
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "GNU Automake".
http://git.sv.gnu.org/gitweb/?p=automake.git;a=commitdiff;h=bc5ba174ea24265b6970e9b747c2cf791f203dc2
The branch, test-protocols has been updated
via bc5ba174ea24265b6970e9b747c2cf791f203dc2 (commit)
via 956c58ab490f79be1829ac4fc3a0da58a4b30973 (commit)
via 445674ff92cb0929f21627dec0ced4bd05a1c5c8 (commit)
from 9a3268ca7a4ec184f014a322ee1ffcbf7055dfd1 (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
commit bc5ba174ea24265b6970e9b747c2cf791f203dc2
Author: Stefano Lattarini <address@hidden>
Date: Fri Aug 5 10:17:01 2011 +0200
docs: some fixlets in documentation on testsuites support
* doc/automake.texi (Script-based Testsuites): Rename node ...
(Scripts-based Testsuites): ... to this. Break overly long lines
in an example (were causing problems with PDF output).
(Parallel Test Harness): Don't intend paragraphs that come after
examples or bulleted list.
(Overview of Custom Test Drivers Support): Fix typo s/the the/the/.
(Declaring Custom Test Drivers): When giving an aside, prefer comma
to parentheses, as it disrupts the reading flow less.
(API for Custom Test Drivers): Remove a sentence that was basically
duplicated from the previous subsection.
(Command-line arguments for test drivers): Don't intend paragraph
coming after an itemized list. Clarify wording and fix grammaros.
Add a couple of cross-references to earlier explanations of `.log'
and `.trs' files.
(Links and external resources): Node renamed ...
(Links and external resources on TAP): ... to this, for clarity.
Slightly extend and clarify introductory sentence.
commit 956c58ab490f79be1829ac4fc3a0da58a4b30973
Author: Stefano Lattarini <address@hidden>
Date: Fri Aug 5 09:28:17 2011 +0200
test harness: be aware of more metadata, simplify test drivers
* lib/am/check.am ($(TEST_SUITE_LOG)): When producing the
global test log, take into account the new metadata field
`:global-test-result:, to write nicely formatted RST section
titles, instead of leaving this chore to the individual test
drivers.
(am_rst_section): Re-introduce this variable, as removed in
commit `v1.11-870-ga27c9c4'.
* lib/test-driver, lib/tap-driver: Write the global test result
as metadata in the `.trs' file, not as (part of) a formatted RST
subsection title in the `.log' file. Related simplifications;
in particular, get rid of temporary files usage.
* doc/automake.texi (Log files generation and test results
recording): Document the new metadata.
* tests/check12.test: Update.
* tests/parallel-tests-harderror.test: Likewise.
* tests/parallel-tests-interrupt.test: Likewise.
* tests/parallel-tests-log-override-2.test: Likewise.
* tests/parallel-tests-log-override-recheck.test: Likewise.
* tests/parallel-tests-unreadable.test: Likewise.
* tests/tap-global-result.test: Likewise.
* tests/test-metadata-results.test: Likewise.
* tests/test-log.test: Likewise, and another minor unrelated fix.
* tests/test-metadata-global-result.test: New test.
* tests/Makefile.am (TESTS): Update.
commit 445674ff92cb0929f21627dec0ced4bd05a1c5c8
Author: Stefano Lattarini <address@hidden>
Date: Wed Aug 3 20:22:50 2011 +0200
tap: check that also a trailing TAP plan can hold a skip directive
* tests/tap-skip-whole-lastline.test: New test.
* tests/Makefile.am (tap_with_common_setup_tests): Add it.
-----------------------------------------------------------------------
Summary of changes:
ChangeLog | 55 +++++++
doc/automake.texi | 81 ++++++----
lib/Automake/tests/Makefile.in | 26 ++--
lib/am/check.am | 35 +++--
lib/tap-driver | 21 +--
lib/test-driver | 22 +--
tests/Makefile.am | 2 +
tests/Makefile.in | 28 +++--
tests/check12.test | 28 +++-
tests/parallel-tests-harderror.test | 6 +-
tests/parallel-tests-interrupt.test | 10 +-
tests/parallel-tests-log-override-2.test | 5 +-
tests/parallel-tests-log-override-recheck.test | 30 ++--
tests/parallel-tests-unreadable.test | 4 +-
tests/tap-global-log.test | 2 +-
tests/tap-global-result.test | 46 ++++--
tests/tap-skip-whole-lastline.test | 59 +++++++
tests/test-log.test | 25 +++-
tests/test-metadata-global-result.test | 196 ++++++++++++++++++++++++
tests/test-metadata-results.test | 37 +++--
20 files changed, 545 insertions(+), 173 deletions(-)
create mode 100755 tests/tap-skip-whole-lastline.test
create mode 100755 tests/test-metadata-global-result.test
diff --git a/ChangeLog b/ChangeLog
index 2d51a28..036e18e 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,58 @@
+2011-08-05 Stefano Lattarini <address@hidden>
+
+ docs: some fixlets in documentation on testsuites support
+ * doc/automake.texi (Script-based Testsuites): Rename node ...
+ (Scripts-based Testsuites): ... to this. Break overly long lines
+ in an example (were causing problems with PDF output).
+ (Parallel Test Harness): Don't intend paragraphs that come after
+ examples or bulleted list.
+ (Overview of Custom Test Drivers Support): Fix typo s/the the/the/.
+ (Declaring Custom Test Drivers): When giving an aside, prefer comma
+ to parentheses, as it disrupts the reading flow less.
+ (API for Custom Test Drivers): Remove a sentence that was basically
+ duplicated from the previous subsection.
+ (Command-line arguments for test drivers): Don't intend paragraph
+ coming after an itemized list. Clarify wording and fix grammaros.
+ Add a couple of cross-references to earlier explanations of `.log'
+ and `.trs' files.
+ (Links and external resources): Node renamed ...
+ (Links and external resources on TAP): ... to this, for clarity.
+ Slightly extend and clarify introductory sentence.
+
+2011-08-04 Stefano Lattarini <address@hidden>
+
+ test harness: be aware of more metadata, simplify test drivers
+ * lib/am/check.am ($(TEST_SUITE_LOG)): When producing the
+ global test log, take into account the new metadata field
+ `:global-test-result:, to write nicely formatted RST section
+ titles, instead of leaving this chore to the individual test
+ drivers.
+ (am_rst_section): Re-introduce this variable, as removed in
+ commit `v1.11-870-ga27c9c4'.
+ * lib/test-driver, lib/tap-driver: Write the global test result
+ as metadata in the `.trs' file, not as (part of) a formatted RST
+ subsection title in the `.log' file. Related simplifications;
+ in particular, get rid of temporary files usage.
+ * doc/automake.texi (Log files generation and test results
+ recording): Document the new metadata.
+ * tests/check12.test: Update.
+ * tests/parallel-tests-harderror.test: Likewise.
+ * tests/parallel-tests-interrupt.test: Likewise.
+ * tests/parallel-tests-log-override-2.test: Likewise.
+ * tests/parallel-tests-log-override-recheck.test: Likewise.
+ * tests/parallel-tests-unreadable.test: Likewise.
+ * tests/tap-global-result.test: Likewise.
+ * tests/test-metadata-results.test: Likewise.
+ * tests/test-log.test: Likewise, and another minor unrelated fix.
+ * tests/test-metadata-global-result.test: New test.
+ * tests/Makefile.am (TESTS): Update.
+
+2011-08-03 Stefano Lattarini <address@hidden>
+
+ tap: check that also a trailing TAP plan can hold a skip directive
+ * tests/tap-skip-whole-lastline.test: New test.
+ * tests/Makefile.am (tap_with_common_setup_tests): Add it.
+
2011-08-03 Stefano Lattarini <address@hidden>
maintcheck: fix more maintainer-check failures
diff --git a/doc/automake.texi b/doc/automake.texi
index 5eb40d5..cab0fca 100644
--- a/doc/automake.texi
+++ b/doc/automake.texi
@@ -326,7 +326,7 @@ Support for test suites
Simple Tests
-* Script-based Testsuites:: Automake-specific concepts and terminology
+* Scripts-based Testsuites:: Automake-specific concepts and terminology
* Serial Test Harness:: Older (and obsolescent) serial test harness
* Parallel Test Harness:: Generic concurrent test harness
@@ -335,7 +335,7 @@ Using the TAP test protocol
* Introduction to TAP::
* Use TAP with the Automake test harness::
* Incompatibilities with other TAP parsers and drivers::
-* Links and external resources::
+* Links and external resources on TAP::
Custom Test Drivers
@@ -8795,13 +8795,13 @@ terminology)?
@section Simple Tests
@menu
-* Script-based Testsuites:: Automake-specific concepts and terminology
+* Scripts-based Testsuites:: Automake-specific concepts and terminology
* Serial Test Harness:: Older (and obsolescent) serial test harness
* Parallel Test Harness:: Generic concurrent test harness
@end menu
address@hidden Script-based Testsuites
address@hidden Script-based Testsuites
address@hidden Scripts-based Testsuites
address@hidden Scripts-based Testsuites
If the special variable @code{TESTS} is defined, its value is taken to be
a list of programs or scripts to run in order to do the testing. Under
@@ -8931,9 +8931,9 @@ HP-UX's @command{/bin/sh},
@c Keep in sync with tests-environment-backcompat.test.
@example
AM_TESTS_ENVIRONMENT = \
-## Some environment initializations are kept in a separate shell file
-## `tests-env.sh', which can make it easier to also run tests from
-## the command line.
+## Some environment initializations are kept in a separate shell
+## file `tests-env.sh', which can make it easier to also run tests
+## from the command line.
. $(srcdir)/tests-env.sh; \
## On Solaris, prefer more POSIX-compliant versions of the standard
## tools by default.
@@ -8941,10 +8941,10 @@ AM_TESTS_ENVIRONMENT = \
PATH=/usr/xpg4/bin:$$PATH; export PATH; \
fi;
@c $$ restore font-lock
-## With this, the test scripts will be able to print diagnostic messages
-## to the original standard error stream, even if the test driver
-## redirects the stderr of the test scripts to a log file before executing
-## them.
+## With this, the test scripts will be able to print diagnostic
+## messages to the original standard error stream, even if the test
+## driver redirects the stderr of the test scripts to a log file
+## before executing them.
AM_TESTS_FD_REDIRECT = 9>&2
@end example
@@ -9129,6 +9129,7 @@ AM_LOG_FLAGS = -Mstrict -w
@trindex check-html
@vindex RST2HTML
@vindex TEST_SUITE_HTML
address@hidden
With @code{make check-html}, the log files may be converted from RST
(reStructuredText, see @uref{http://docutils.sourceforge.net/@/rst.html})
to HTML using @samp{RST2HTML}, which defaults to @command{rst2html} or
@@ -9204,6 +9205,7 @@ does the same as @code{recheck} but again converts the
resulting
@file{.log} file in HTML format, like the @code{check-html} target.
@end itemize
address@hidden
In order to guarantee an ordering between tests even with @code{make
address@hidden, dependencies between the corresponding @file{.log} files
may be specified through usual @command{make} dependencies. For example,
@@ -9295,7 +9297,7 @@ protocols are TAP and SubUnit).
It's very important to note that, even when using custom test drivers,
most of the infrastructure described in the previous section about the
-the parallel harness remains in place; this includes:
+parallel harness remains in place; this includes:
@itemize
@item
@@ -9361,9 +9363,9 @@ coexist.
The developer-reserved variable @code{AM_LOG_DRIVER_FLAGS} and the
user-reserved variable @code{LOG_DRIVER_FLAGS} can be used to define
-flags that will be passed to each invocation of @code{LOG_DRIVER}
-(with the user-defined flags obviously taking precedence over the
-developer-reserved ones). Similarly, for each extension @var{ext}
+flags that will be passed to each invocation of @code{LOG_DRIVER},
+with the user-defined flags obviously taking precedence over the
+developer-reserved ones. Similarly, for each extension @var{ext}
declared in @code{TEST_EXTENSIONS}, flags listed in
@address@hidden and
@address@hidden will be passed to
@@ -9378,9 +9380,7 @@ for new features or to satisfy additional portability
requirements.
The main characteristic of these APIs is that they are designed to share
as much infrastructure, semantics, and implementation details as possible
-with the parallel test harness and its default driver. So everything
-said in the previous section should apply here too, unless explicitly
-stated otherwise.
+with the parallel test harness and its default driver.
@menu
* Command-line arguments for test drivers::
@@ -9409,13 +9409,15 @@ suffix and a directory component (as in e.g.,
@file{sub/foo.test}), and is
mostly meant to be used in console reports about testsuite advancements and
results (@pxref{Testsuite progress output}).
@item address@hidden@var{PATH}.log}
-The @file{.log} file the test driver must create. If it has a directory
-component (as in e.g., @file{sub/foo.log}), the test harness will ensure
-that such directory exist @emph{before} the test driver is called.
+The @file{.log} file the test driver must create (@pxref{Basics of
+test metadata}). If it has a directory component (as in e.g.,
address@hidden/foo.log}), the test harness will ensure that such directory
+exists @emph{before} the test driver is called.
@item address@hidden@var{PATH}.trs}
-The @file{.trs} file the test driver must create. If it has a directory
-component (as in e.g., @file{sub/foo.trs}), the test harness will ensure
-that such directory exist @emph{before} the test driver is called.
+The @file{.trs} file the test driver must create (@pxref{Basics of
+test metadata}). If it has a directory component (as in e.g.,
address@hidden/foo.trs}), the test harness will ensure that such directory
+exists @emph{before} the test driver is called.
@item address@hidden|address@hidden
Whether the console output should be colorized or not (@pxref{Simple
tests and color-tests}, to learn when this option gets activated and
@@ -9431,9 +9433,10 @@ conventions in use.
Explicitly terminate the list of options.
@end table
-The first of the remaining arguments passed to the test driver is the
-program to be run, and the other arguments are command-line options
-and arguments for this program.
address@hidden
+The first non-option argument passed to the test driver is the program to
+be run, and all the following ones are command-line options and arguments
+for this program.
Note that the exact semantics attached to the @option{--color-tests},
@option{--expect-failure} and @option{--enable-hard-errors} options are
@@ -9508,6 +9511,19 @@ just a waste of space in normal situations, e.g., when a
test script is
successful. What happens when two or more @code{:copy-in-global-log:}
fields are present in the same @file{.trs} file is undefined behaviour.
address@hidden Keep in sync with 'test-metadata-global-result.test'.
address@hidden @code{:test-global-result:}
address@hidden :test-global-result:
address@hidden reStructuredText field, @code{:test-global-result:}
+This is used to declare the "global result" of the script. Currently,
+the value of this field is needed only to be reported (more or less
+verbatim) in the generated global log file @code{$(TEST_SUITE_LOG)},
+so it's quite free-form. For example, a test script which run 10 test
+cases, 6 of which pass and 4 of which are skipped, could reasonably have
+a @code{PASS/SKIP} value for this field, while a test script which run
+19 successful tests and one failed test could have an @code{ALMOST
+PASSED} value. What happens when two or more @code{:test-global-result:}
+fields are present in the same @file{.trs} file is undefined behaviour.
@end table
@noindent
@@ -9576,7 +9592,7 @@ harness should remain untouched, and continue to work
correctly.
* Introduction to TAP::
* Use TAP with the Automake test harness::
* Incompatibilities with other TAP parsers and drivers::
-* Links and external resources::
+* Links and external resources on TAP::
@end menu
@node Introduction to TAP
@@ -9748,12 +9764,13 @@ the ``hard error'' concept of the default
@option{parallel-tests} driver.
@emph{TODO}: there's surely something else ...
@end itemize
address@hidden Links and external resources
address@hidden Links and external resources
address@hidden Links and external resources on TAP
address@hidden Links and external resources on TAP
@noindent
Here are some links to more extensive official or third-party
-documentation and resources:
+documentation and resources about the TAP protocol and related
+tools and libraries.
@itemize @bullet
@item
@uref{http://search.cpan.org/~petdance/Test-Harness/lib/Test/Harness/TAP.pod,
diff --git a/lib/Automake/tests/Makefile.in b/lib/Automake/tests/Makefile.in
index 78a7e4a..c305321 100644
--- a/lib/Automake/tests/Makefile.in
+++ b/lib/Automake/tests/Makefile.in
@@ -113,8 +113,9 @@ am__nobase_list = $(am__nobase_strip_setup); \
am__base_list = \
sed '$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;s/\n/ /g' | \
sed '$$!N;$$!N;$$!N;$$!N;s/\n/ /g'
-# Restructured Text title.
+# Restructured Text title and section.
am__rst_title = sed 's/.*/ & /;h;s/./=/g;p;x;p;g;p;s/.*//'
+am__rst_section = sed 'p;s/./=/g;p;g'
# Solaris 10 'make', and several other traditional 'make' implementations,
# pass "-e" to $(SHELL), and POSIX 2008 even requires this. Work around it
# by disabling -e (using the XSI extension "set +e") if it's set.
@@ -417,15 +418,20 @@ $(TEST_SUITE_LOG): $(TEST_LOGS)
echo ".. contents:: :depth: 2"; \
echo; \
for b in $$bases; do \
- if grep "^$$ws*:copy-in-global-log:$$ws*no$$ws*$$" $$b.trs \
- >/dev/null; then :; \
- elif test ! -r $$b.log; then \
- echo "ERROR: cannot read $$b.log" >&2; \
- echo && echo "WARNING: could not read $$b.log!"; \
- else \
- echo && cat $$b.log; \
- fi; \
- done; \
+ if grep "^$$ws*:copy-in-global-log:$$ws*no$$ws*$$" $$b.trs \
+ >/dev/null; then continue; \
+ fi; \
+ glob_res=`sed -n -e "s/$$ws*$$//" \
+ -e "s/^$$ws*:global-test-result:$$ws*//p" \
+ $$b.trs`; \
+ test -n "$$glob_res" || glob_res=RUN; \
+ echo "$$glob_res: $$b" | $(am__rst_section); \
+ if test ! -r $$b.log; then \
+ echo "fatal: making $@: $$b.log is unreadable" >&2; \
+ exit 1; \
+ fi; \
+ cat $$b.log; echo; \
+ done; \
} >$(TEST_SUITE_LOG).tmp; \
mv $(TEST_SUITE_LOG).tmp $(TEST_SUITE_LOG); \
if $$success; then \
diff --git a/lib/am/check.am b/lib/am/check.am
index c406e9f..36adef8 100644
--- a/lib/am/check.am
+++ b/lib/am/check.am
@@ -75,8 +75,9 @@ include inst-vars.am
## test is XFAIL or not. You can disable this feature by setting the
## variable DISABLE_HARD_ERRORS to a nonempty value.
-# Restructured Text title.
+# Restructured Text title and section.
am__rst_title = sed 's/.*/ & /;h;s/./=/g;p;x;p;g;p;s/.*//'
+am__rst_section = sed 'p;s/./=/g;p;g'
# Solaris 10 'make', and several other traditional 'make' implementations,
# pass "-e" to $(SHELL), and POSIX 2008 even requires this. Work around it
@@ -240,18 +241,26 @@ $(TEST_SUITE_LOG): $(TEST_LOGS)
echo; \
for b in $$bases; do \
## FIXME: one fork per test -- this is horrendously inefficient!
- if grep "^$$ws*:copy-in-global-log:$$ws*no$$ws*$$" $$b.trs \
- >/dev/null; then :; \
-## If we cannot read the .log file of a test ...
- elif test ! -r $$b.log; then \
-## ... print an error message to the console ...
- echo "ERROR: cannot read $$b.log" >&2; \
-## ... and a warning in the `test-suite.log' file.
- echo && echo "WARNING: could not read $$b.log!"; \
- else \
- echo && cat $$b.log; \
- fi; \
- done; \
+ if grep "^$$ws*:copy-in-global-log:$$ws*no$$ws*$$" $$b.trs \
+ >/dev/null; then continue; \
+ fi; \
+## Get the declared "global result" of the test.
+## FIXME: yet another one fork per test here!
+ glob_res=`sed -n -e "s/$$ws*$$//" \
+ -e "s/^$$ws*:global-test-result:$$ws*//p" \
+ $$b.trs`; \
+## If no global result is explicitly declared, we'll merely mark the
+## test as "RUN" in the global test log.
+ test -n "$$glob_res" || glob_res=RUN; \
+## Write the name and result of the test as an RST section title.
+ echo "$$glob_res: $$b" | $(am__rst_section); \
+## If we should have remade any unreadable `.log', above.
+ if test ! -r $$b.log; then \
+ echo "fatal: making $@: $$b.log is unreadable" >&2; \
+ exit 1; \
+ fi; \
+ cat $$b.log; echo; \
+ done; \
} >$(TEST_SUITE_LOG).tmp; \
mv $(TEST_SUITE_LOG).tmp $(TEST_SUITE_LOG); \
## Emit the test summary on the console.
diff --git a/lib/tap-driver b/lib/tap-driver
index 3facdfd..14f0507 100755
--- a/lib/tap-driver
+++ b/lib/tap-driver
@@ -169,6 +169,8 @@ TEST_RESULTS :
sub write_test_results ()
{
open RES, ">", $trs_file or die "opening $trs_file: $!\n";
+ print RES ":global-test-result: " .
+ get_global_test_result . "\n";
print RES ":recheck: " .
(must_recheck ? "yes" : "no") . "\n";
print RES ":copy-in-global-log: " .
@@ -185,10 +187,10 @@ sub start (@)
# Redirect stderr and stdout to a temporary log file. Save the
# original stdout stream, since we need it to print testsuite
# progress output.
+ open LOG, ">", $log_file or die "opening $log_file: $!\n";
open OLDOUT, ">&STDOUT" or die "duplicating stdout: $!\n";
- open TMP, ">$log_file-t" or die "opening $log_file-t: $!\n";
- open STDOUT, ">&TMP" or die "redirecting stdout: $!\n";
- open STDERR, ">&TMP" or die "redirecting stderr: $!\n";
+ open STDOUT, ">&LOG" or die "redirecting stdout: $!\n";
+ open STDERR, ">&LOG" or die "redirecting stderr: $!\n";
$parser = TAP::Parser->new ({ exec => address@hidden, merge => $cfg{merge}
});
$parser->ignore_exit(1) if $cfg{"ignore-exit"};
}
@@ -196,20 +198,7 @@ sub start (@)
sub finish ()
{
write_test_results;
-
- open LOG, ">", $log_file or die "opening $log_file: $!\n";
- my $global_result = get_global_test_result;
- my $global_result_line = "$global_result: $test_script_name";
- print LOG "$global_result_line\n";
- print LOG "=" x length ($global_result_line) . "\n";
- print LOG "\n";
-
- close TMP or die "closing $log_file-t: $!\n";
- # FIXME: remove this hack!
- my $test_output = `cat $log_file-t && rm -f $log_file-t`;
- print LOG $test_output;
close LOG or die "closing $log_file: $!\n";
-
exit 0;
}
diff --git a/lib/test-driver b/lib/test-driver
index 472ba49..c1645a2 100755
--- a/lib/test-driver
+++ b/lib/test-driver
@@ -1,7 +1,7 @@
#! /bin/sh
# test-driver - basic driver script for the `parallel-tests' mode.
-scriptversion=2011-08-01.21; # UTC
+scriptversion=2011-08-04.22; # UTC
# Copyright (C) 2011 Free Software Foundation, Inc.
#
@@ -55,9 +55,6 @@ The \`--test-name' and \`--log-file' options are mandatory.
END
}
-# Restructured Text section.
-rst_section () { sed 'p;s/./=/g;p;g'; }
-
# TODO: better error handling in option parsing (in particular, ensure
# TODO: $logfile, $trsfile and $test_name are defined).
test_name= # Used for reporting.
@@ -93,16 +90,14 @@ else
red= grn= lgn= blu= mgn= std=
fi
-tmpfile=$logfile-t
-do_exit='rm -f $tmpfile; (exit $st); exit $st'
+do_exit='rm -f $logfile; (exit $st); exit $st'
trap "st=129; $do_exit" 1
trap "st=130; $do_exit" 2
trap "st=141; $do_exit" 13
trap "st=143; $do_exit" 15
-rm -f $tmpfile
# Test script is run here.
-"$@" >$tmpfile 2>&1
+"$@" >$logfile 2>&1
estatus=$?
if test $enable_hard_errors = no && test $estatus -eq 99; then
estatus=1
@@ -121,18 +116,11 @@ esac
echo "${col}${res}${std}: $test_name"
# Register the test result, and other relevant metadata.
-echo ":test-result: $res (exit status: $estatus)" > $trsfile
+echo ":test-result: $res" > $trsfile
+echo ":global-test-result: $res" >> $trsfile
echo ":recheck: $recheck" >> $trsfile
echo ":copy-in-global-log: $gcopy" >> $trsfile
-# Now write log file.
-{
- echo "$res: $test_name (exit: $estatus)" | rst_section
- echo
- cat $tmpfile
-} > $logfile
-rm -f $tmpfile
-
# Local Variables:
# mode: shell-script
# sh-indentation: 2
diff --git a/tests/Makefile.am b/tests/Makefile.am
index b9b75ff..9fdb891 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -766,6 +766,7 @@ test-driver-trs-suffix-registered.test \
test-driver-fail.test \
test-log.test \
test-metadata-global-log.test \
+test-metadata-global-result.test \
test-metadata-recheck.test \
test-metadata-results.test \
test-missing.test \
@@ -1175,6 +1176,7 @@ tap-realtime.test \
tap-recheck-logs.test \
tap-skip-whole-whitespace.test \
tap-skip-whole.test \
+tap-skip-whole-lastline.test \
tap-todo-skip-together.test \
tap-todo-skip-whitespace.test \
tap-todo-skip.test \
diff --git a/tests/Makefile.in b/tests/Makefile.in
index b030585..ee0d378 100644
--- a/tests/Makefile.in
+++ b/tests/Makefile.in
@@ -118,8 +118,9 @@ am__nobase_list = $(am__nobase_strip_setup); \
am__base_list = \
sed '$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;s/\n/ /g' | \
sed '$$!N;$$!N;$$!N;$$!N;s/\n/ /g'
-# Restructured Text title.
+# Restructured Text title and section.
am__rst_title = sed 's/.*/ & /;h;s/./=/g;p;x;p;g;p;s/.*//'
+am__rst_section = sed 'p;s/./=/g;p;g'
# Solaris 10 'make', and several other traditional 'make' implementations,
# pass "-e" to $(SHELL), and POSIX 2008 even requires this. Work around it
# by disabling -e (using the XSI extension "set +e") if it's set.
@@ -1020,6 +1021,7 @@ test-driver-trs-suffix-registered.test \
test-driver-fail.test \
test-log.test \
test-metadata-global-log.test \
+test-metadata-global-result.test \
test-metadata-recheck.test \
test-metadata-results.test \
test-missing.test \
@@ -1411,6 +1413,7 @@ tap-realtime.test \
tap-recheck-logs.test \
tap-skip-whole-whitespace.test \
tap-skip-whole.test \
+tap-skip-whole-lastline.test \
tap-todo-skip-together.test \
tap-todo-skip-whitespace.test \
tap-todo-skip.test \
@@ -1567,15 +1570,20 @@ $(TEST_SUITE_LOG): $(TEST_LOGS)
echo ".. contents:: :depth: 2"; \
echo; \
for b in $$bases; do \
- if grep "^$$ws*:copy-in-global-log:$$ws*no$$ws*$$" $$b.trs \
- >/dev/null; then :; \
- elif test ! -r $$b.log; then \
- echo "ERROR: cannot read $$b.log" >&2; \
- echo && echo "WARNING: could not read $$b.log!"; \
- else \
- echo && cat $$b.log; \
- fi; \
- done; \
+ if grep "^$$ws*:copy-in-global-log:$$ws*no$$ws*$$" $$b.trs \
+ >/dev/null; then continue; \
+ fi; \
+ glob_res=`sed -n -e "s/$$ws*$$//" \
+ -e "s/^$$ws*:global-test-result:$$ws*//p" \
+ $$b.trs`; \
+ test -n "$$glob_res" || glob_res=RUN; \
+ echo "$$glob_res: $$b" | $(am__rst_section); \
+ if test ! -r $$b.log; then \
+ echo "fatal: making $@: $$b.log is unreadable" >&2; \
+ exit 1; \
+ fi; \
+ cat $$b.log; echo; \
+ done; \
} >$(TEST_SUITE_LOG).tmp; \
mv $(TEST_SUITE_LOG).tmp $(TEST_SUITE_LOG); \
if $$success; then \
diff --git a/tests/check12.test b/tests/check12.test
index 4be4b80..27d61f6 100755
--- a/tests/check12.test
+++ b/tests/check12.test
@@ -34,10 +34,12 @@ END
cat > a.test << 'END'
#!/bin/sh
+echo a.test: exit ${A_EXIT_STATUS-0}
exit ${A_EXIT_STATUS-0}
END
cat > b.test << 'END'
#!/bin/sh
+echo b.test: exit ${B_EXIT_STATUS-0}
exit ${B_EXIT_STATUS-0}
END
chmod +x a.test b.test
@@ -135,9 +137,9 @@ for vpath in : false; do
test -f spanner.sum
# This checks will be run only by the autogenerated `check12-p.test'.
if test x"$parallel_tests" = x"yes"; then
- cat test-suite.log
- grep '^PASS: a\.test (exit: 0)' a.log
- grep '^PASS: b\.test (exit: 0)' b.log
+ test -f test-suite.log
+ test -f a.log
+ test -f b.log
else
:
fi
@@ -170,9 +172,13 @@ for vpath in : false; do
# This checks will be run only by the autogenerated `check12-p.test'.
if test x"$parallel_tests" = x"yes"; then
cat test-suite.log
- grep '^PASS: a\.test (exit: 0)' a.log
- grep '^FAIL: b\.test (exit: 1)' b.log
- grep '^FAIL: b\.test (exit: 1)' test-suite.log
+ cat a.log
+ cat b.log
+ grep '^a\.test: exit 0$' a.log
+ grep '^b\.test: exit 1$' b.log
+ grep '^FAIL: b$' test-suite.log
+ grep '^b\.test: exit 1$' test-suite.log
+ grep '^a\.test' test-suite.log && Exit 1
else :; fi
CHECKLOCAL_EXIT_STATUS=1 $MAKE check && Exit 1
@@ -189,9 +195,13 @@ for vpath in : false; do
# This checks will be run only by the autogenerated `check12-p.test'.
if test x"$parallel_tests" = x"yes"; then
cat test-suite.log
- grep '^PASS: a\.test (exit: 0)' a.log
- grep '^FAIL: b\.test (exit: 23)' b.log
- grep '^FAIL: b\.test (exit: 23)' test-suite.log
+ cat a.log
+ cat b.log
+ grep '^a\.test: exit 0$' a.log
+ grep '^b\.test: exit 23$' b.log
+ grep '^FAIL: b$' test-suite.log
+ grep '^b\.test: exit 23$' test-suite.log
+ grep '^a\.test' test-suite.log && Exit 1
else :; fi
grep 'check-local failed :-(' local.log
diff --git a/tests/parallel-tests-harderror.test
b/tests/parallel-tests-harderror.test
index f30fd52..2092e05 100755
--- a/tests/parallel-tests-harderror.test
+++ b/tests/parallel-tests-harderror.test
@@ -69,13 +69,13 @@ DISABLE_HARD_ERRORS=x $MAKE check
$MAKE check DISABLE_HARD_ERRORS='' && Exit 1
cat test-suite.log
-grep '^ERROR: foo\.test .*exit.*99' test-suite.log
+grep '^ERROR: foo$' test-suite.log
cd sub
# The `-e' is wanted here.
DISABLE_HARD_ERRORS='' $MAKE -e check && Exit 1
cat test-suite.log
-grep '^ERROR: bar\.test .*exit.*99' test-suite.log
+grep '^ERROR: bar$' test-suite.log
cd ..
# Check the distributions.
@@ -90,7 +90,7 @@ $AUTOMAKE Makefile
./config.status Makefile
VERBOSE=yes $MAKE check && Exit 1
grep '^FAIL' test-suite.log && Exit 1
-grep '^ERROR: bar\.test .*exit.*99' sub/test-suite.log
+grep '^ERROR: bar$' sub/test-suite.log
echo 'DISABLE_HARD_ERRORS = zardoz' >> sub/Makefile
VERBOSE=yes $MAKE check
diff --git a/tests/parallel-tests-interrupt.test
b/tests/parallel-tests-interrupt.test
index 1c5d9ce..a7c1ba4 100755
--- a/tests/parallel-tests-interrupt.test
+++ b/tests/parallel-tests-interrupt.test
@@ -47,8 +47,8 @@ cat > foo.test << 'END'
exec 2>&9
echo "foo is starting to run"
ls -l >&2
-cat foo.log-t >&2 || : > fail
-grep '^foo is starting to run$' foo.log-t >&2 || : > fail
+cat foo.log >&2 || : > fail
+grep '^foo is starting to run$' foo.log >&2 || : > fail
cat pid >&2 || : > fail
kill -$signum `cat pid` || : > fail
END
@@ -65,17 +65,15 @@ $AUTOMAKE
trapped_signals='1 2 13 15'
for signum in $trapped_signals; do
- rm -f pid fail *.log *.log-t
+ rm -f pid fail *.log
env signum=$signum $MAKE check && { ls -l; Exit 1; }
ls -l
# These files shouldn't exist, but in case they do, their content might
# provide helpful information about the causes of the failure(s).
- cat foo.log-t || :
cat foo.log || :
cat test-suite.log || :
test -f fail && Exit 1
- test -f foo.log-t && Exit 1
- test -f foo.log && Exit 1
+ ls | $EGREP 'foo.*\.(log|tmp)' && Exit 1
done
:
diff --git a/tests/parallel-tests-log-override-2.test
b/tests/parallel-tests-log-override-2.test
index dfd6f2f..54d16c3 100755
--- a/tests/parallel-tests-log-override-2.test
+++ b/tests/parallel-tests-log-override-2.test
@@ -77,9 +77,10 @@ do
test ! -f fail.log
grep '^PASS: pass\.test$' stdout
grep '^SKIP: skip\.test$' stdout
- $FGREP 'skip.test' partial.log
+ $FGREP 'SKIP: skip' partial.log
$FGREP '% skipped test %' partial.log
- $EGREP '(pass2|skip2|fail)\.test' stdout partial.log && Exit 1
+ $EGREP '(pass2|skip2|fail)\.test' stdout && Exit 1
+ $EGREP '(pass2|skip2|fail)' partial.log && Exit 1
rm -f *.log
done
diff --git a/tests/parallel-tests-log-override-recheck.test
b/tests/parallel-tests-log-override-recheck.test
index d8cab8a..db73718 100755
--- a/tests/parallel-tests-log-override-recheck.test
+++ b/tests/parallel-tests-log-override-recheck.test
@@ -27,22 +27,22 @@ AC_OUTPUT
END
cat > Makefile.am <<'END'
-TESTS = foo.test bar.test baz.test
+TESTS = foofoo.test barbar.test bazbaz.test
END
-cat > foo.test <<'END'
+cat > foofoo.test <<'END'
#! /bin/sh
echo "this is $0"
exit 0
END
-cat > bar.test <<'END'
+cat > barbar.test <<'END'
#! /bin/sh
echo "this is $0"
exit 99
END
-cat > baz.test <<'END'
+cat > bazbaz.test <<'END'
#! /bin/sh
echo "this is $0"
exit ${BAZ_EXIT_STATUS-1}
@@ -67,9 +67,9 @@ cat stdout
ls -l
count_test_results total=2 pass=0 fail=1 skip=0 xfail=0 xpass=0 error=1
for x in stdout my.log; do
- $FGREP foo.test $x && Exit 1
- $FGREP bar.test $x
- $FGREP baz.test $x
+ $FGREP foofoo $x && Exit 1
+ $FGREP barbar $x
+ $FGREP bazbaz $x
done
chmod a-rw my.log
@@ -78,15 +78,15 @@ BAZ_EXIT_STATUS=0 TEST_SUITE_LOG=my2.log $MAKE -e recheck
>stdout \
cat stdout
ls -l
count_test_results total=2 pass=1 fail=0 skip=0 xfail=0 xpass=0 error=1
-$FGREP foo.test stdout && Exit 1
-$FGREP bar.test stdout
-$FGREP baz.test stdout
-$FGREP foo.test my2.log && Exit 1
-$FGREP bar.test my2.log
-$FGREP baz.test my2.log && Exit 1
+$FGREP foofoo stdout && Exit 1
+$FGREP barbar stdout
+$FGREP bazbaz stdout
+$FGREP foofoo my2.log && Exit 1
+$FGREP barbar my2.log
+$FGREP bazbaz my2.log && Exit 1
chmod u+r test-suite.log my.log
-$FGREP baz.test test-suite.log
-$FGREP baz.test my.log
+$FGREP bazbaz test-suite.log
+$FGREP bazbaz my.log
:
diff --git a/tests/parallel-tests-unreadable.test
b/tests/parallel-tests-unreadable.test
index 47edc9a..6192a13 100755
--- a/tests/parallel-tests-unreadable.test
+++ b/tests/parallel-tests-unreadable.test
@@ -67,9 +67,9 @@ for files in \
grep '^:test-result: PASS' foo.trs
grep '^barbarbar$' bar.log
grep '^:test-result: SKIP' bar.trs
- grep '^SKIP: bar.test' test-suite.log
+ grep '^SKIP: bar' test-suite.log
grep '^barbarbar$' test-suite.log
- $EGREP 'foo\.test|foofoofoo' test-suite.log && Exit 1
+ $EGREP ':.*foo|foofoofoo' test-suite.log && Exit 1
done
:
diff --git a/tests/tap-global-log.test b/tests/tap-global-log.test
index a4f6428..d343581 100755
--- a/tests/tap-global-log.test
+++ b/tests/tap-global-log.test
@@ -106,7 +106,7 @@ END
TESTS="`echo *.test`" $MAKE -e check || :
cat test-suite.log
-grep 'ok\.test|not seen' test-suite.log && Exit 1
+grep ':.*ok|not seen' test-suite.log && Exit 1
for s in skip todo fail xpass bail error; do
$FGREP "::$s::" test-suite.log
diff --git a/tests/tap-global-result.test b/tests/tap-global-result.test
index c49fae4..32e2401 100755
--- a/tests/tap-global-result.test
+++ b/tests/tap-global-result.test
@@ -79,26 +79,42 @@ END
# TODO: add scripts with TAP errors (multiple plans, out-of-order
# tests, etc).
-TESTS="`echo *.test`" $MAKE -e check >stdout && { cat stdout; Exit 1; }
+tests=`echo *.test` # Also required later.
+
+TESTS="$tests" $MAKE -e check >stdout && { cat stdout; Exit 1; }
cat stdout
+# Dirty tricks required here.
for tst in ok skip skipall fail fail2 xpass xpass2 error error2 \
hodgepodge hodgepodge-all; do
- sed -e 2q $tst.log > $tst.res
+ echo :copy-in-global-log: yes >> $tst.trs
done
-cat *.res # For debugging.
-
-grep '^PASS:' ok.res
-grep '^SKIP:' skip.res
-grep '^SKIP:' skipall.res
-grep '^FAIL:' fail.res
-grep '^FAIL:' fail2.res
-grep '^FAIL:' xpass.res
-grep '^FAIL:' xpass2.res
-grep '^ERROR:' error.res
-grep '^ERROR:' error2.res
-grep '^ERROR:' hodgepodge.res
-grep '^ERROR:' hodgepodge-all.res
+rm -f test-suite.log
+TESTS="$tests" $MAKE -e test-suite.log && Exit 1
+cat test-suite.log
+
+have_rst_section ()
+{
+ eqeq=`echo "$1" | sed 's/./=/g'`
+ # Assume $1 contains no RE metacharacters.
+ sed -n "/^$1$/,/^$eqeq$/p" test-suite.log > got
+ (echo "$1" && echo "$eqeq") > exp
+ cat exp
+ cat got
+ diff exp got
+}
+
+have_rst_section 'PASS: ok'
+have_rst_section 'SKIP: skip'
+have_rst_section 'SKIP: skipall'
+have_rst_section 'FAIL: fail'
+have_rst_section 'FAIL: fail2'
+have_rst_section 'FAIL: xpass'
+have_rst_section 'FAIL: xpass2'
+have_rst_section 'ERROR: error'
+have_rst_section 'ERROR: error2'
+have_rst_section 'ERROR: hodgepodge'
+have_rst_section 'ERROR: hodgepodge-all'
:
diff --git a/tests/tap-skip-whole-lastline.test
b/tests/tap-skip-whole-lastline.test
new file mode 100755
index 0000000..25697e0
--- /dev/null
+++ b/tests/tap-skip-whole-lastline.test
@@ -0,0 +1,59 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+# - the special plan format to skip all the tests in a script can also
+# be used on the last line, even if preceded by non-TAP output, TAP
+# diagnostic, and even non-failing TAP test results.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+echo AM_TEST_LOG_DRIVER_FLAGS = --comments >> Makefile
+
+cat > foo.test <<END
+some non-TAP text, will be copied in the global log
+1..0 # SKIP from the last line
+END
+
+cat > bar.test <<END
+# some TAP diagnostic, will go to console
+1..0
+END
+
+cat > baz.test <<END
+ok 1 - seen also if the whole test is skipped!
+1..0 # SKIP
+END
+
+st=0
+TESTS='foo.test bar.test baz.test' $MAKE -e check >stdout || st=$?
+cat stdout
+cat test-suite.log
+test $st -eq 0
+
+grep '^SKIP: foo\.test .* from the last line$' stdout
+grep '^SKIP: bar\.test$' stdout
+grep '^SKIP: baz\.test$' stdout
+grep '^# bar\.test: some TAP diagnostic, will go to console$' stdout
+grep '^some non-TAP text, will be copied in the global log$' test-suite.log
+grep '^PASS: baz\.test 1 - seen also if the whole test is skipped!$' stdout
+test `grep -c ': .*\.test' stdout` -eq 4
+count_test_results total=4 pass=1 fail=0 xpass=0 xfail=0 skip=3 error=0
+
+:
diff --git a/tests/test-log.test b/tests/test-log.test
index 1385386..c25cb20 100755
--- a/tests/test-log.test
+++ b/tests/test-log.test
@@ -50,7 +50,7 @@ cat > skip.test <<END
#! /bin/sh
echo "$pmarker skip $pmarker"
echo "# $cmarker skip $cmarker"
-exit 1
+exit 77
END
cat > xfail.test <<END
@@ -111,14 +111,25 @@ done
test `$FGREP -c "$pmarker" my.log` -eq 5
test `$FGREP -c "$cmarker" my.log` -eq 5
+have_rst_section ()
+{
+ eqeq=`echo "$1" | sed 's/./=/g'`
+ # Assume $1 contains no RE metacharacters.
+ sed -n "/^$1$/,/^$eqeq$/p" $2 > got
+ (echo "$1" && echo "$eqeq") > exp
+ cat exp
+ cat got
+ diff exp got
+}
+
# Passed test scripts shouldn't be mentioned in the global log.
-$EGREP '(^pass|[^x]pass)\.test' my.log && Exit 1
+$EGREP ':.*[^x]pass' my.log && Exit 1
# But failing (expectedly or not) and skipped ones should.
-$FGREP 'xfail.test' my.log
-$FGREP 'skip.test' my.log
-$FGREP 'fail.test' my.log
-$FGREP 'xpass.test' my.log
-$FGREP 'error.test' my.log
+have_rst_section 'SKIP: skip' my.log
+have_rst_section 'FAIL: fail' my.log
+have_rst_section 'XFAIL: xfail' my.log
+have_rst_section 'XPASS: xpass' my.log
+have_rst_section 'ERROR: error' my.log
touch error2.log test-suite.log global.log
TEST_SUITE_LOG=my.log $MAKE -e mostlyclean
diff --git a/tests/test-metadata-global-result.test
b/tests/test-metadata-global-result.test
new file mode 100755
index 0000000..d20b249
--- /dev/null
+++ b/tests/test-metadata-global-result.test
@@ -0,0 +1,196 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Parallel testsuite harness: check APIs for the registering the
+# "global test result" in `*.trs' files, as documented in the automake
+# manual.
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+cat >> configure.in << 'END'
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+TEST_EXTENSIONS = .test .x
+TEST_LOG_DRIVER = ./dummy-driver
+X_LOG_DRIVER = ./dummy-driver
+TESTS = foo.test zar-doz.test
+END
+
+cat > dummy-driver <<'END'
+#! /bin/sh
+set -e; set -u
+while test $# -gt 0; do
+ case $1 in
+ --log-file) log_file=$2; shift;;
+ --trs-file) trs_file=$2; shift;;
+ --test-name) test_name=$2; shift;;
+ --expect-failure|--color-tests|--enable-hard-errors) shift;;
+ --) shift; break;;
+ *) echo "$0: invalid option/argument: '$1'" >&2; exit 2;;
+ esac
+ shift
+done
+echo logloglog > $log_file
+cp $1 $trs_file
+END
+chmod a+x dummy-driver
+
+# Do this in a subroutine to avoid quoting problem in the backticked
+# command substitution below.
+get_escaped_line()
+{
+ sed -e 's,[$^/\\\.],\\&,g' -e 1q "$@"
+}
+
+have_result ()
+{
+ cat > exp; echo >> exp; echo logloglog >> exp
+ eline=`get_escaped_line exp`
+ sed -n -e "/^$eline$/,/^logloglog$/p" test-suite.log > got
+ cat exp; cat got
+ diff exp got
+}
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE
+
+./configure
+
+: Basic checks.
+
+echo :global-test-result: PASS > foo.test
+echo :global-test-result: ERROR > zar-doz.x
+
+$MAKE check
+cat test-suite.log
+
+have_result <<END
+PASS: foo
+=========
+END
+
+have_result <<END
+ERROR: zar-doz
+==============
+END
+
+: Try usage documented in the manual.
+
+echo :global-test-result: PASS/SKIP > foo.test
+echo :global-test-result: ALMOST PASSED > zar-doz.x
+
+$MAKE check
+cat test-suite.log
+
+have_result <<END
+PASS/SKIP: foo
+==============
+END
+
+have_result <<END
+ALMOST PASSED: zar-doz
+======================
+END
+
+: Fields ':test-result:' does not interfere with the global test result.
+
+cat > foo.test << 'END'
+:test-result: FAIL
+:global-test-result: PASS
+:test-result: ERROR
+END
+
+cat > zar-doz.x << 'END'
+:global-test-result: FAIL
+:test-result: SKIP
+:test-result: XFAIL
+END
+
+$MAKE check && Exit 1
+cat test-suite.log
+
+have_result <<END
+PASS: foo
+=========
+END
+
+have_result <<END
+FAIL: zar-doz
+=============
+END
+
+: What happens when ':global-test-result:' is absent.
+
+cat > foo.test << 'END'
+:test-result: PASS
+:test-result: ERROR
+END
+: > zar-doz.x
+
+$MAKE check && Exit 1
+cat test-suite.log
+
+have_result <<END
+RUN: foo
+========
+END
+
+have_result <<END
+RUN: zar-doz
+============
+END
+
+# Leading and trailing whitespace gets eaten/normalized.
+
+echo ":global-test-result:SKIP${tab} ${tab}${tab}" > foo.test
+echo ":global-test-result:${tab} ${tab}XFAIL ${tab} " > zar-doz.x
+
+$MAKE check
+cat test-suite.log
+
+have_result <<END
+SKIP: foo
+=========
+END
+
+have_result <<END
+XFAIL: zar-doz
+==============
+END
+
+# Whitespaces before and after `:global-test-result:' are handled OK.
+
+echo " $tab:global-test-result:PASS" > foo.test
+echo "${tab}${tab}:global-test-result:${tab} ${tab}SKIP" > zar-doz.x
+
+$MAKE check
+cat test-suite.log
+
+have_result <<END
+PASS: foo
+=========
+END
+
+have_result <<END
+SKIP: zar-doz
+=============
+END
+
+:
diff --git a/tests/test-metadata-results.test b/tests/test-metadata-results.test
index effa488..923f88f 100755
--- a/tests/test-metadata-results.test
+++ b/tests/test-metadata-results.test
@@ -84,36 +84,43 @@ $AUTOMAKE
./configure
-# Basic checks. Also that that "old-style" directives with format
-# "RESULT: test-name" are be ignored now.
+# Basic checks. Also that that `:global-test-result:' fields and
+# "old-style" directives with format "RESULT: test-name" are now ignored.
+
+: > foo.test
+echo blah blah blah > bar.test
+mk_check
+count_test_results total=0 pass=0 fail=0 xpass=0 xfail=0 skip=0 error=0
+
+cat > foo.test <<END
+:test-global-result: PASS
+:test-result: FAIL
+END
+cat > bar.test <<END
+:test-result: SKIP
+:test-global-result: ERROR
+END
+mk_check && Exit 1
+count_test_results total=2 pass=0 fail=1 xpass=0 xfail=0 skip=1 error=0
cat > foo.test <<END
FAIL: foo.test
:test-result: PASS
+:test-global-result: XPASS
END
echo ERROR: bar.test > bar.test
mk_check
count_test_results total=1 pass=1 fail=0 xpass=0 xfail=0 skip=0 error=0
cat > foo.test <<END
-PASS: foo.test
-:test-result: FAIL
-END
-echo SKIP: bar.test > bar.test
-: > bar.test
-mk_check && Exit 1
-count_test_results total=1 pass=0 fail=1 xpass=0 xfail=0 skip=0 error=0
-
-cat > foo.test <<END
-PASS: foo.test
+:test-global-result: SKIP
:test-result: FAIL
END
cat > bar.test <<END
-ERROR: foo.test
-:test-result: SKIP
+:test-global-result: PASS
END
mk_check && Exit 1
-count_test_results total=2 pass=0 fail=1 xpass=0 xfail=0 skip=1 error=0
+count_test_results total=1 pass=0 fail=1 xpass=0 xfail=0 skip=0 error=0
cat > foo.test <<END
:test-result: XFAIL
hooks/post-receive
--
GNU Automake
| [Prev in Thread] |
Current Thread |
[Next in Thread] |
- [Automake-commit] [SCM] GNU Automake branch, test-protocols, updated. v1.11-960-gbc5ba17,
Stefano Lattarini <=