[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Automake-NG] [PATCH 01/11] [ng] check: big refactoring with semantic ch
|
From: |
Stefano Lattarini |
|
Subject: |
[Automake-NG] [PATCH 01/11] [ng] check: big refactoring with semantic changes in parallel-tests support |
|
Date: |
Wed, 9 May 2012 11:30:06 +0200 |
In this change, we move some further processing related to the parallel
testsuite harness support from Automake runtime to make runtime.
A welcome collateral effect of this is that we are now able to cope
with test scripts whose name ends with several concatenated suffixes.
Another (less welcome) collateral effect of the present change is that
the user now cannot override TEST_LOGS at make runtime anymore:
# This won't work anymore!
make check TESTS_LOGS="foo.log bar.log baz.log"
This admittedly is a small(ish) regression. But it's not a real problem,
since we now explicitly allow *and document* that TESTS can be overridden
at runtime without bothering to add the test suffixes to its entries:
# This will work even if complete names of the tests are (say)
# "foo.test", "bar.test$(EXEEXT)" and "baz$(EXEEXT)".
make check TESTS="foo bar baz"
With such usage, the user don't have to bother knowing which the exact
extensions of each tests are -- which was precisely the only advantage
of overriding TEST_LOGS instead of TESTS.
* NG-NEWS: Update.
* doc/automake.texi: Likewise.
* lib/am/header-vars.am (am__strip_suffixes_0, am__strip_suffixes): New
internal macros ...
* t/internals.tap: ... covered by new checks in this test, and used ...
* lib/am/check.am (am__xfail_test_bases): ... to define this new internal
variable in term of $(am__cooked_xfail_tests) (and thus eventually of
$(XFAIL_TESTS)), and to ...
(am__TEST_BASES): ... (re)define this pre-existing internal variable in
terms of $(am__cooked_tests) (and thus eventually of $(TESTS)), rather
than in terms of $(TEST_LOGS) as was done before.
(am__TEST_RESULTS, am__TEST_LOGS): Define in terms of $(am__TEST_BASES).
(TEST_LOGS): Define to $(am__TEST_LOGS), for backward-compatibility.
($(TEST_SUITE_LOG)): Depend on $(am__TEST_RESULTS) rather than on
$(TEST_LOGS).
(am__check_pre): Use $(am__xfail_test_bases) and new temporary shell
variable $f2, instead of $(am__cooked_xfail_tests) and $f, to decide
whether a test is expected to fail or not.
Fix some imprecise or botched comments since we are at it.
(check-TESTS, recheck): When re-invoking make recursively, override
am__TEST_BASES rather than TEST_LOGS. Relate tweaks.
* automke.in (handle_tests): Pass new transform %HANDLE-EXEEXT% when
including check.am (used for the definitions of am__xfail_test_bases
and am__TEST_BASES). Don't define the TEST_LOGS make variable, nor the
auxiliary make variables am__test_logs1, am__test_logs2, etc. Put
$(am__TEST_LOGS) and $(am__TEST_RESULTS) among the "mostlyclean" files
(rather than "$(TEST_LOGS)" and "$(TEST_LOGS:.log=.trs)" as before).
* t/parallel-tests-concatenated-suffix.sh: New test.
* t/nodeps.sh: Delete, it was causing too much spurious failures.
* t/color2.sh: Avoid a spurious failure by not calling make with
the '-e' flag.
* t/parallel-tests-exeext.sh: Extended to check the user is not
forced to specify the test suffixes nor the $(EXEEXT) suffix
when overriding TESTS on the command line.
* t/check-concurrency-bug9245.sh: Adjust to the new semantics of "no
TEST_LOGS overriding from the command line".
* t/parallel-tests-cmdline-override.sh: Likewise.
* t/parallel-tests-dynamic.sh: Likewise.
* t/parallel-tests-fork-bomb.sh: Likewise.
* t/parallel-tests-log-override-2.sh: Likewise.
* t/tap-basic.sh: Likewise.
* t/test-driver-custom-multitest-recheck2.sh: Likewise.
* t/test-missing.sh: Likewise.
* t/test-trs-basic.sh: Likewise.
* t/test-trs-recover.sh: Likewise.
* t/parallel-tests-empty-tests.sh: Likewise, and extended to check
that an empty TESTS produces am__cooked_tests, am__TEST_RESULTS,
am__TEST_BASES and am__TEST_LOGS that are empty as well.
* t/parallel-tests.sh: Adjust to the new semantics of "no TEST_LOGS
overriding from the command line". Remove a workaround required only
by HP-UX make. Make some "FIXME"-style comments stick out.
* t/serial-tests.sh: Adjust some checks that grep the Automake
generated Makefile.in.
Signed-off-by: Stefano Lattarini <address@hidden>
---
NG-NEWS | 14 ++++
automake.in | 27 ++-----
doc/automake.texi | 43 +++++------
lib/am/check.am | 50 +++++++------
lib/am/header-vars.am | 19 +++++
t/check-concurrency-bug9245.sh | 2 +-
t/color2.sh | 2 +-
t/internals.tap | 60 ++++++++++++++--
t/nodep.sh | 40 -----------
t/parallel-tests-cmdline-override.sh | 12 ++--
t/parallel-tests-concatenated-suffix.sh | 76 ++++++++++++++++++++
t/parallel-tests-dynamic.sh | 12 ++--
...y-testlogs.sh => parallel-tests-empty-tests.sh} | 10 ++-
t/parallel-tests-exeext.sh | 11 +++
t/parallel-tests-fork-bomb.sh | 3 +-
t/parallel-tests-log-override-2.sh | 8 +--
t/parallel-tests.sh | 16 ++---
t/serial-tests.sh | 4 +-
t/tap-basic.sh | 6 +-
t/test-driver-custom-multitest-recheck2.sh | 25 +++----
t/test-missing.sh | 6 +-
t/test-trs-basic.sh | 14 ++--
t/test-trs-recover.sh | 4 +-
23 files changed, 283 insertions(+), 181 deletions(-)
delete mode 100755 t/nodep.sh
create mode 100755 t/parallel-tests-concatenated-suffix.sh
rename t/{parallel-tests-empty-testlogs.sh => parallel-tests-empty-tests.sh}
(90%)
diff --git a/NG-NEWS b/NG-NEWS
index 99cb0bc..da2fb20 100644
--- a/NG-NEWS
+++ b/NG-NEWS
@@ -104,6 +104,20 @@ Parallel testsuite harness
work as expected:
TESTS = $(wildcard $(srcdir)/t[0-9][0-9]*.sh)
+* It is not anymore possible to override TEST_LOGS at make runtime as a
+ mean to redefine the list of tests to be run.
+
+ # This won't work anymore!
+ make check TESTS_LOGS="foo.log bar.log baz.log"
+
+ But it's still possible do so overriding TESTS at make runtime; and
+ in this case (as was the case for TEST_LOGS overriding), it is not
+ necessary to specify the extensions of the tests to be run:
+
+ # This will work even if complete names of the tests are (say)
+ # "foo.test", "bar.test$(EXEEXT)" and "baz$(EXEEXT)".
+ make check TESTS="foo bar baz"
+
Miscellaneous
=============
diff --git a/automake.in b/automake.in
index 483bb79..e735359 100644
--- a/automake.in
+++ b/automake.in
@@ -4757,11 +4757,12 @@ sub handle_tests
if (var ('TESTS'))
{
push (@check_tests, 'check-TESTS');
- my $check_deps = "@check";
$output_rules .= &file_contents ('check', new Automake::Location,
COLOR => !! option 'color-tests',
PARALLEL_TESTS => !! option
'parallel-tests',
- CHECK_DEPS => $check_deps);
+ CHECK_DEPS => "@check",
+ 'HANDLE-EXEEXT' => (exists
$configure_vars{'EXEEXT'} ?
+ 'TRUE' : 'FALSE'));
if (my $parallel_tests = option 'parallel-tests')
{
@@ -4792,30 +4793,12 @@ sub handle_tests
# an explicit 'all' target) rather than from the 'all.test' script,
# thus causing all sort of mishaps and confusion.
push @test_suffixes, '';
-
- define_variable ('am__test_logs1',
- '$(patsubst %,%.log,$(strip $(am__cooked_tests)))',
- INTERNAL);
- my $nhelper = 1;
foreach my $test_suffix (@test_suffixes)
{
handle_per_suffix_test ($test_suffix);
- my @cooked_test_suffixes = ($test_suffix);
- push @cooked_test_suffixes, ($test_suffix . '$(EXEEXT)')
- if exists $configure_vars{'EXEEXT'};
- foreach my $cooked_test_suffix (@cooked_test_suffixes)
- {
- my $old_var = 'am__test_logs' . $nhelper++;
- my $cur_var = 'am__test_logs' . $nhelper;
- define_variable (
- $cur_var,
- "\$($old_var:$cooked_test_suffix.log=.log)",
- INTERNAL);
- }
}
- define_variable ('TEST_LOGS', "\$(am__test_logs$nhelper)", INTERNAL);
- $clean_files{'$(TEST_LOGS)'} = MOSTLY_CLEAN;
- $clean_files{'$(TEST_LOGS:.log=.trs)'} = MOSTLY_CLEAN;
+ $clean_files{'$(am__TEST_LOGS)'} = MOSTLY_CLEAN;
+ $clean_files{'$(am__TEST_RESULTS)'} = MOSTLY_CLEAN;
$clean_files{'$(TEST_SUITE_LOG)'} = MOSTLY_CLEAN;
}
else
diff --git a/doc/automake.texi b/doc/automake.texi
index 2890a8c..e45c5da 100644
--- a/doc/automake.texi
+++ b/doc/automake.texi
@@ -8996,14 +8996,13 @@ For best results, the tests should be verbose by
default now.
@vindex TEST_LOGS
Each couple of @file{.log} and @file{.trs} files is created when the
corresponding test has completed. The set of log files is listed in
-the read-only variable @code{TEST_LOGS}, and defaults to @code{TESTS},
-with the executable extension if any (@pxref{EXEEXT}), as well as any
-suffix listed in @code{TEST_EXTENSIONS} removed, and @file{.log} appended.
-Results are undefined if a test file name ends in several concatenated
-suffixes. @code{TEST_EXTENSIONS} defaults to @file{.test}; it can be
-overridden by the user, in which case any extension listed in it must be
-constituted by a dot, followed by a non-digit alphabetic character,
-followed by any number of alphabetic characters.
+the @emph{read-only} variable @code{TEST_LOGS}, and defaults to
address@hidden, with the executable extension if any (@pxref{EXEEXT}),
+as well as any suffix listed in @code{TEST_EXTENSIONS} removed, and
address@hidden appended. @code{TEST_EXTENSIONS} defaults to @file{.test};
+it can be overridden by the user, in which case any extension listed
+in it must be constituted by a dot, followed by a non-digit alphabetic
+character, followed by any number of alphabetic characters.
@c Keep in sync with test-extensions.sh
For example, @samp{.sh}, @samp{.T} and @samp{.t1} are valid extensions,
while @samp{.x-y}, @samp{.6c} and @samp{.t.1} are not.
@@ -9075,10 +9074,17 @@ You can set the @code{TESTS} variable. For example,
you can use a
command like this to run only a subset of the tests:
@example
-make check TESTS="foo.test bar.test"
+make check TESTS="foo.test bar.sh"
@end example
-Note however that the command above will unconditionally overwrite the
+If @code{.test} and @code{.sh} are in @code{$(TEST_EXTENSIONS)}, the
+command above can also be shortened as:
+
address@hidden
+make check TESTS="foo bar"
address@hidden example
+
+Note however that the commands above will unconditionally overwrite the
@file{test-suite.log} file, thus clobbering the recorded results
of any previous testsuite run. This might be undesirable for packages
whose testsuite takes long time to execute. Luckily, this problem can
@@ -9094,19 +9100,8 @@ will write the result of the partial testsuite runs to
the
@file{partial.log}, without touching @file{test-suite.log}.
@item
-You can set the @code{TEST_LOGS} variable. By default, this variable is
-computed at @command{make} run time from the value of @code{TESTS} as
-described above. For example, you can use the following:
-
address@hidden
-set x subset*.log; shift
-make check TEST_LOGS="foo.log $*"
address@hidden example
-
-The comments made above about @code{TEST_SUITE_LOG} overriding applies
-here too.
-
address@hidden
address@hidden FIXME: this should become "RECHECK_TESTS", for consistency.
address@hidden FIXME: this will imply a change to the API, of course.
@vindex RECHECK_LOGS
@cindex lazy test execution
By default, the test harness removes all old per-test @file{.log} and
@@ -9217,7 +9212,7 @@ parallel harness remains in place; this includes:
@itemize
@item
list of test scripts defined in @code{TESTS}, and overridable at
-runtime through the redefinition of @code{TESTS} or @code{TEST_LOGS};
+runtime through the redefinition of @code{TESTS};
@item
concurrency through the use of @command{make}'s option @option{-j};
@item
diff --git a/lib/am/check.am b/lib/am/check.am
index 8d5aa40..ae40c00 100644
--- a/lib/am/check.am
+++ b/lib/am/check.am
@@ -202,12 +202,10 @@ am__common_driver_flags = \
--expect-failure "$$am__expect_failure"
# To be inserted before the command running the test. Creates the
-# directory for the log if needed. Stores in $dir the directory
-# containing $f, in $tst the test, in $log the log. Executes the
-# developer- defined test setup AM_TESTS_ENVIRONMENT (if any), and
-# passes TESTS_ENVIRONMENT. Set up options for the wrapper that
-# will run the test scripts (or their associated LOG_COMPILER, if
-# thy have one).
+# directory for the log if needed. Executes the developer-defined
+# test setup AM_TESTS_ENVIRONMENT (if any), and passes TESTS_ENVIRONMENT.
+# Set up options for the wrapper that will run the test scripts (or their
+# associated LOG_COMPILER, if they have one).
am__check_pre = \
$(am__sh_e_setup); \
$(am__tty_colors); \
@@ -225,24 +223,34 @@ if test -n '$(DISABLE_HARD_ERRORS)'; then \
else \
am__enable_hard_errors=yes; \
fi; \
-case " $(am__cooked_xfail_tests) " in \
- *" $$f "*) am__expect_failure=yes;; \
- *) am__expect_failure=no;; \
+f2='$(patsubst $(srcdir)/%,%,$*)'; \
+case " $(am__xfail_test_bases) " in \
+ *" $$f2 "*) am__expect_failure=yes;; \
+ *) am__expect_failure=no;; \
esac; \
$(AM_TESTS_ENVIRONMENT) $(TESTS_ENVIRONMENT)
# The names of the tests scripts with any registered extension removed
-# (i.e., equivalently, the names of the test logs, with the '.log' suffix
-# stripped), and the name of the test result files (i.e., equivalently, the
-# names of the test logs, with the '.log' suffix substituted by the '.trs'
-# suffix). This honors runtime overriding of TESTS and TEST_LOGS. It
-# also works around the GNU make 3.80 bug where trailing whitespace in
-# "TESTS = foo.test $(empty)" causes $(TESTS_LOGS) to erroneously expand
-# to "foo.log .log".
-am__TEST_BASES = $(patsubst %.log,%,$(strip $(TEST_LOGS)))
-am__TEST_RESULTS = $(patsubst %.log,%.trs,$(strip $(TEST_LOGS)))
+# This honors runtime overriding of TESTS, and takes into account
+# $(EXEEXT) appending.
+am__TEST_BASES = $(call am__strip_suffixes, $(TEST_EXTENSIONS), \
+?!HANDLE-EXEEXT?$(am__cooked_tests))
+?HANDLE-EXEEXT?$(patsubst %$(EXEEXT),%,$(am__cooked_tests)))
+# Likewise for the xfailing tests.
+am__xfail_test_bases = $(call am__strip_suffixes, $(TEST_EXTENSIONS), \
+?!HANDLE-EXEEXT?$(am__cooked_xfail_tests))
+?HANDLE-EXEEXT?$(patsubst %$(EXEEXT),%,$(am__cooked_xfail_tests)))
-$(TEST_SUITE_LOG): $(TEST_LOGS) $(am__TEST_RESULTS)
+# The $(strip) is to work around the GNU make 3.80 bug where trailing
+# whitespace in "TESTS = foo.test $(empty)" causes $(TESTS_LOGS) to
+# erroneously expand to "foo.log .log".
+am__TEST_RESULTS = $(addsuffix .trs,$(strip $(am__TEST_BASES)))
+am__TEST_LOGS = $(addsuffix .log,$(strip $(am__TEST_BASES)))
+
+# $(TEST_LOGS) is a published interface.
+TEST_LOGS = $(am__TEST_LOGS)
+
+$(TEST_SUITE_LOG): $(am__TEST_LOGS) $(am__TEST_RESULTS)
@$(am__sh_e_setup); $(am__tty_colors); \
fatal () { echo "fatal: making $@: $$*" >&2; exit 1; }; \
## Detect a possible circular dependency, and error out.
@@ -394,6 +402,8 @@ check-TESTS recheck:
## unexpected outcome (FAIL or XPASS) in the earlier run.
bases=`for i in $$bases; do echo $$i; done \
| $(am__list_recheck_tests)` || exit 1; \
+## Remove newlines and normalize whitespace.
+ bases=`echo $$bases`; \
fi; \
log_list=`for i in $$bases; do echo $$i.log; done`; \
trs_list=`for i in $$bases; do echo $$i.trs; done`; \
@@ -407,7 +417,7 @@ check-TESTS recheck:
test -z "$$log_list" || rm -f $$log_list; \
test -z "$$trs_list" || rm -f $$trs_list; \
fi; \
- $(MAKE) $(TEST_SUITE_LOG) TEST_LOGS="$$log_list"
+ $(MAKE) $(TEST_SUITE_LOG) am__TEST_BASES="$$bases"
## Recheck must depend on $(check_SCRIPTS), $(check_PROGRAMS), etc.
## It must also depend on the 'all' target. See automake bug#11252.
diff --git a/lib/am/header-vars.am b/lib/am/header-vars.am
index a081639..1d6571c 100644
--- a/lib/am/header-vars.am
+++ b/lib/am/header-vars.am
@@ -71,6 +71,25 @@ am__uniq = $(strip \
$(am__empty), \
$(call am__lastword,$(1)))))
+## $(call am__strip_suffixes_0, SUFFIXES, WORD)
+## --------------------------------------------
+## Strip any of the SUFFIXES from WORD. Even if WORD terminates with
+## several suffixes, only one is stripped: the first one that matches.
+am__strip_suffixes_0 = $(strip \
+ $(if $(strip $(1)), \
+ $(if $(filter %$(firstword $(1)), $(2)), \
+ $(patsubst %$(firstword $(1)), %, $(2)), \
+ $(call am__strip_suffixes_0, \
+ $(call am__strip_firstword, $(1)), $(2))), \
+ $(2)))
+
+## $(call am__strip_suffixes, SUFFIXES, LIST)
+## ------------------------------------------
+## Strip any of the SUFFIXES from each of the entries of LIST. Even if an
+## entry of LIST terminates with several suffixes, only one is stripped:
+## the first one that matches.
+am__strip_suffixes = \
+ $(foreach am__i,$(2),$(call am__strip_suffixes_0,$(1),$(am__i)))
## Some derived variables that have been found to be useful.
pkgdatadir = $(datadir)/@PACKAGE@
diff --git a/t/check-concurrency-bug9245.sh b/t/check-concurrency-bug9245.sh
index c2d3d53..87fcade 100755
--- a/t/check-concurrency-bug9245.sh
+++ b/t/check-concurrency-bug9245.sh
@@ -49,7 +49,7 @@ for j in '' -j1 -j2; do
$MAKE $j TESTS=foo.test check && Exit 1
if test x"$am_parallel_tests" = x"yes"; then
$MAKE $j recheck && Exit 1
- $MAKE $j TEST_LOGS=foo.log check && Exit 1
+ $MAKE $j TESTS=foo check && Exit 1
rm -f test-suite.log
$MAKE $j test-suite.log && Exit 1
test -f test-suite.log || Exit 1
diff --git a/t/color2.sh b/t/color2.sh
index d03c6a2..4e6862a 100755
--- a/t/color2.sh
+++ b/t/color2.sh
@@ -107,7 +107,7 @@ test_no_color ()
}
cat >expect-make <<'END'
-eval spawn $env(MAKE) -e check
+eval spawn $env(MAKE) check
expect eof
END
diff --git a/t/internals.tap b/t/internals.tap
index 6efc96f..6d69166 100755
--- a/t/internals.tap
+++ b/t/internals.tap
@@ -19,7 +19,7 @@
am_create_testdir=empty
. ./defs || Exit 1
-plan_ 3
+plan_ 5
cp "$am_amdir"/header-vars.am . \
|| fatal_ "fetching makefile fragment headers-vars.am"
@@ -29,6 +29,8 @@ cp "$am_amdir"/header-vars.am . \
LC_ALL=C $EGREP -v '(^##|address@hidden@)' header-vars.am > defn.mk
rm -f header-vars.am
+# WARNING: there are a lot of embedded tabs in this makefile.
+# DO NOT "NORMALIZE" THEM TO SPACES!
cat > Makefile << 'END'
include ./defn.mk
@@ -72,10 +74,60 @@ test-uniq:
test '$(call am__uniq,3 1 1 4 1 4 1 1)' = '3 1 4'
test '$(call am__uniq, 1 3 1 )' = '1 3'
+.PHONY: test-strip-suffixes0
+test-strip-suffixes0:
+ test '$(call am__strip_suffixes_0,,)' = ''
+ test '$(call am__strip_suffixes_0, ,)' = ''
+ test '$(call am__strip_suffixes_0,, )' = ''
+ test '$(call am__strip_suffixes_0, , )' = ''
+ test '$(call am__strip_suffixes_0,x,)' = ''
+ test '$(call am__strip_suffixes_0,x, )' = ''
+ test '$(call am__strip_suffixes_0,x y, )' = ''
+ test '$(call am__strip_suffixes_0,,x)' = 'x'
+ test '$(call am__strip_suffixes_0, ,x)' = 'x'
+ test '$(call am__strip_suffixes_0,.c,foo.c)' = 'foo'
+ test '$(call am__strip_suffixes_0,.c .c++, bar.c++)' = 'bar'
+ test '$(call am__strip_suffixes_0,.c .c++, bar.cxx)' = 'bar.cxx'
+ test '$(call am__strip_suffixes_0,x,ax)' = 'a'
+ test '$(call am__strip_suffixes_0,x,xa)' = 'xa'
+ test '$(call am__strip_suffixes_0,x,xx)' = 'x'
+ test '$(call am__strip_suffixes_0,x,xux)' = 'xu'
+ test '$(call am__strip_suffixes_0, .a .b, x.a.a)' = 'x.a'
+ test '$(call am__strip_suffixes_0, .a .b, x.a.b)' = 'x.a'
+ test '$(call am__strip_suffixes_0, .a .b, x.b.a)' = 'x.b'
+ test '$(call am__strip_suffixes_0, .a .b, x.b.b)' = 'x.b'
+ # Corner cases: the *first* matched suffix is stripped
+ test '$(call am__strip_suffixes_0, .a .b.a, foo.b.a)' = 'foo.b'
+ test '$(call am__strip_suffixes_0, .b.a .a, foo.b.a)' = 'foo'
+
+.PHONY: test-strip-suffixes
+test-strip-suffixes:
+ test '$(call am__strip_suffixes,,)' = ''
+ test '$(call am__strip_suffixes, ,)' = ''
+ test '$(call am__strip_suffixes,, )' = ''
+ test '$(call am__strip_suffixes, , )' = ''
+ test '$(call am__strip_suffixes,x,)' = ''
+ test '$(call am__strip_suffixes,x y, )' = ''
+ test '$(call am__strip_suffixes,,x y)' = 'x y'
+ test '$(call am__strip_suffixes, ,x y)' = 'x y'
+ test '$(call am__strip_suffixes,.c,foo.c)' = 'foo'
+ test '$(call am__strip_suffixes,.foo,a.foo b.foo)' = 'a b'
+ test '$(call am__strip_suffixes, x y, ax ay ax)' = 'a a a'
+ test '$(call am__strip_suffixes, .c .c++, \
+ foo.c bar.c++ baz.cxx zap.c)' = 'foo bar baz.cxx zap'
+ test '$(call am__strip_suffixes, .a .b, \
+ 1.a.a 2.a.b 3.b.a 4.b.b)' = '1.a 2.a 3.b 4.b'
+ # Corner cases: the *first* matched suffix is stripped
+ test '$(call am__strip_suffixes, .a .b.a, foo.b.a bar.a)' \
+ = 'foo.b bar'
+ test '$(call am__strip_suffixes, .b.a .a, foo.b.a bar.a)' \
+ = 'foo bar'
END
-command_ok_ "am__strip_firstword" $MAKE test-strip-firstword
-command_ok_ "am__strip_lastword" $MAKE test-strip-lastword
-command_ok_ "am__uniq" $MAKE test-uniq
+command_ok_ am__strip_firstword $MAKE test-strip-firstword
+command_ok_ am__strip_lastword $MAKE test-strip-lastword
+command_ok_ am__uniq $MAKE test-uniq
+command_ok_ am__test_strip_suffixes_0 $MAKE test-strip-suffixes0
+command_ok_ am__test_strip_suffixes $MAKE test-strip-suffixes
:
diff --git a/t/nodep.sh b/t/nodep.sh
deleted file mode 100755
index 1002c54..0000000
--- a/t/nodep.sh
+++ /dev/null
@@ -1,40 +0,0 @@
-#! /bin/sh
-# Copyright (C) 1999-2012 Free Software Foundation, Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2, or (at your option)
-# any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program. If not, see <http://www.gnu.org/licenses/>.
-
-# Test to make sure no-dependencies option does the right thing.
-# Bug report from Greg A. Woods.
-
-. ./defs || Exit 1
-
-cat > Makefile.am << 'END'
-AUTOMAKE_OPTIONS = no-dependencies
-bin_PROGRAMS = zardoz
-zardoz_SOURCES = y.c
-END
-
-cat >> configure.ac << 'END'
-AC_PROG_CC
-END
-
-mkdir x
-
-: > y.c
-
-$ACLOCAL
-$AUTOMAKE
-
-$FGREP -v '$(filter --%,' Makefile.in | grep '%' && Exit 1
-Exit 0
diff --git a/t/parallel-tests-cmdline-override.sh
b/t/parallel-tests-cmdline-override.sh
index 7d219e9..5dcd93a 100755
--- a/t/parallel-tests-cmdline-override.sh
+++ b/t/parallel-tests-cmdline-override.sh
@@ -14,8 +14,8 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Check that we can use indirections when overriding TESTS and
-# TEST_LOGS from the command line.
+# Check that we can use indirections when overriding TESTS from
+# the command line.
am_parallel_tests=yes
. ./defs || Exit 1
@@ -73,12 +73,12 @@ do_check ()
return $st
}
-tests='a.t $(var1) $(var3:.d=.t) $(var4:=.test)'
-test_logs='a.log $(var1:.test=.log) $(var3:.d=.log) $(var4:=.log)'
+tests1='a.t $(var1) $(var3:.d=.t) $(var4:=.test)'
+tests2='a $(var1:.test=) $(var3:.d=) $(var4)'
touch a.t b.test c.test d.t e.test
-do_check TESTS="$tests"
-do_check TEST_LOGS="$test_logs"
+do_check TESTS="$tests1"
+do_check TESTS="$tests2"
:
diff --git a/t/parallel-tests-concatenated-suffix.sh
b/t/parallel-tests-concatenated-suffix.sh
new file mode 100755
index 0000000..0791222
--- /dev/null
+++ b/t/parallel-tests-concatenated-suffix.sh
@@ -0,0 +1,76 @@
+#! /bin/sh
+# Copyright (C) 2012 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# The parallel-tests driver should be able to cope with test scripts
+# whose names end with several concatenated suffixes.
+
+am_parallel_tests=yes
+. ./defs || Exit 1
+
+cat >> configure.ac << 'END'
+AC_OUTPUT
+END
+
+tests='foo.sh foo.t.sh foo.sh.t foo.x.x foo.x.t.sh foo.t.x.sh foo.sh.t.x'
+
+for t in $tests; do
+ (echo '#!/bin/sh' && echo 'echo == /$0 ==') > $t
+ chmod a+x $t
+done
+
+cat > Makefile.am <<END
+TEST_EXTENSIONS = .t .sh .x
+TESTS = $tests
+END
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+./configure
+
+for j in '' -j4; do
+
+ # Use append mode here to avoid dropping output. See automake bug#11413.
+ # Also, use 'echo' here to "nullify" the previous contents of 'stdout',
+ # since Solaris 10 /bin/sh would try to optimize a ':' away after the
+ # first iteration, even if it is redirected.
+ echo " " >stdout
+ $MAKE $j check >>stdout || { cat stdout; Exit 1; }
+ cat stdout
+ count_test_results total=7 pass=7 fail=0 skip=0 xfail=0 xpass=0 error=0
+ for t in $tests; do grep "^PASS: $t *$" stdout; done
+
+ grep '== .*/foo\.sh ==' foo.log
+ grep '== .*/foo\.t\.sh ==' foo.t.log
+ grep '== .*/foo\.sh\.t ==' foo.sh.log
+ grep '== .*/foo\.x\.x ==' foo.x.log
+ grep '== .*/foo\.x\.t\.sh ==' foo.x.t.log
+ grep '== .*/foo\.t\.x\.sh ==' foo.t.x.log
+ grep '== .*/foo\.sh\.t\.x ==' foo.sh.t.log
+
+ $MAKE $j clean
+ test ! -f foo.log
+ test ! -f foo.t.log
+ test ! -f foo.sh.log
+ test ! -f foo.x.log
+ test ! -f foo.x.t.log
+ test ! -f foo.t.x.log
+ test ! -f foo.sh.t.log
+
+done
+
+:
diff --git a/t/parallel-tests-dynamic.sh b/t/parallel-tests-dynamic.sh
index 3b4fcb9..a18662c 100755
--- a/t/parallel-tests-dynamic.sh
+++ b/t/parallel-tests-dynamic.sh
@@ -14,8 +14,7 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Check that dynamic content for $(TESTS) and $(TEST_LOGS) is
-# supported.
+# Check that dynamic content for $(TESTS) is supported.
am_parallel_tests=yes
. ./defs || Exit 1
@@ -124,16 +123,15 @@ grep '^XFAIL: t98S\.sh' stdout
$MAKE mostlyclean
test "`find . -name *.log`" = ./config.log
-# We can also override $(TEST_LOGS) dynamically.
-$MAKE check TEST_LOGS='$(shell echo t00 | sed "s/$$/-foo.log/") t99.log'
+$MAKE check TESTS='$(shell echo t00 | sed "s/$$/-foo/") t99'
test -f t00-foo.log
test -f t99.log
# A little tricky in that we rely on the .log files created by
# the previous run to be present.
-$MAKE check TEST_LOGS="\
- \$(wildcard t[0-9]*.log) \
- \$(call my_add_dirprefix, t, nosuffix).log \
+$MAKE check TESTS="\
+ \$(patsubst %.log,%,\$(wildcard t[0-9]*.log)) \
+ \$(call my_add_dirprefix, t, nosuffix) \
" > stdout || { cat stdout; Exit 1; }
cat stdout
diff --git a/t/parallel-tests-empty-testlogs.sh
b/t/parallel-tests-empty-tests.sh
similarity index 90%
rename from t/parallel-tests-empty-testlogs.sh
rename to t/parallel-tests-empty-tests.sh
index c49a412..018cce0 100755
--- a/t/parallel-tests-empty-testlogs.sh
+++ b/t/parallel-tests-empty-tests.sh
@@ -14,9 +14,7 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Check parallel-tests features:
-# - empty TESTS
-# - empty TEST_LOGS
+# Check parallel-tests features: empty TESTS
am_parallel_tests=yes
. ./defs || Exit 1
@@ -35,7 +33,9 @@ mkdir sub1 sub2
cat > sub1/Makefile.am << 'END'
TESTS =
check-local:
- echo $(TEST_LOGS) | grep . && exit 1; exit 0
+ echo $(am__cooked_tests) $(am__TEST_LOGS) \
+ $(am__TEST_RESULTS) $(am__TEST_BASES) \
+ | grep . && exit 1; exit 0
END
cat > sub2/Makefile.am << 'END'
@@ -74,8 +74,6 @@ for vpath in : false; do
cd ../sub2
$MAKE check VERBOSE=yes TESTS=''
no_test_has_run
- $MAKE check VERBOSE=yes TEST_LOGS=''
- no_test_has_run
cd ..
$MAKE check
cat sub2/foo.log
diff --git a/t/parallel-tests-exeext.sh b/t/parallel-tests-exeext.sh
index ac94533..14cb753 100755
--- a/t/parallel-tests-exeext.sh
+++ b/t/parallel-tests-exeext.sh
@@ -61,4 +61,15 @@ test ! -r y.bin.log
test -f b.log
test ! -r b.test.log
+# Opportunistically check that we are not forced to specify
+# the test suffixes nor the $(EXEEXT) suffix when overriding
+# TESTS on the command line.
+rm -f *.log *.trs
+
+$MAKE check TESTS='y a b'
+ls -l # For debugging.
+test -f a.log
+test -f b.log
+test -f y.log
+
:
diff --git a/t/parallel-tests-fork-bomb.sh b/t/parallel-tests-fork-bomb.sh
index 193a6e7..5029d2c 100755
--- a/t/parallel-tests-fork-bomb.sh
+++ b/t/parallel-tests-fork-bomb.sh
@@ -75,8 +75,7 @@ rm -f *.log *.test
: > 2.test
: > 3.test
: > foobar.test
-do_check foobar.log TEST_LOGS='0.log 1.log foobar.log 2.log 3.log' \
- TEST_SUITE_LOG=foobar.log
+do_check foobar.log TESTS='0 1 foobar 2 3' TEST_SUITE_LOG=foobar.log
rm -f *.log *.test
:
diff --git a/t/parallel-tests-log-override-2.sh
b/t/parallel-tests-log-override-2.sh
index 3c11c9c..ab83943 100755
--- a/t/parallel-tests-log-override-2.sh
+++ b/t/parallel-tests-log-override-2.sh
@@ -14,9 +14,7 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Check parallel-tests features: runtime redefinition of:
-# - $(TEST_SUITE_LOG) and $(TESTS)
-# - $(TEST_SUITE_LOG) and $(TEST_LOGS)
+# parallel-tests: runtime redefinition of $(TEST_SUITE_LOG) and $(TESTS)
am_parallel_tests=yes
. ./defs || Exit 1
@@ -60,8 +58,8 @@ $AUTOMAKE -a
# Extra quoting for the sake of Solaris sh bugs.
for test_list_override in \
'TESTS=pass.test skip.test' \
- 'TEST_LOGS=pass.log skip.log'
-do
+ 'TESTS=pass skip' \
+; do
eval "\$MAKE TEST_SUITE_LOG=partial.log '$test_list_override'" \
check >stdout || { cat stdout; Exit 1; }
cat stdout
diff --git a/t/parallel-tests.sh b/t/parallel-tests.sh
index 4861ff5..63949a2 100755
--- a/t/parallel-tests.sh
+++ b/t/parallel-tests.sh
@@ -19,8 +19,7 @@
# - log files, and what goes in 'test-suite.log'
# - make clean
# - dependencies between tests
-# - TESTS redefinition at runtime
-# - TEST_LOGS redefinition at runtime
+# - TESTS redefinition at runtime (with and without test suffixes)
# - RECHECK_LOGS redefinition at runtime
am_parallel_tests=yes
@@ -37,19 +36,14 @@ foo.log: bar.log
bar.log: baz.log
END
-# foo.test and bar.test sleep to ensure their logs are always strictly newer
-# than the logs of their prerequisites, for HP-UX make. The quoting pleases
-# maintainer-check.
cat > foo.test <<'END'
#! /bin/sh
echo "this is $0"
-sleep '1'
exit 0
END
cat > bar.test <<'END'
#! /bin/sh
echo "this is $0"
-sleep '1'
exit 99
END
cat > baz.test <<'END'
@@ -85,9 +79,9 @@ test ! -f test-suite.log
# Check dependencies: baz.test needs to run before bar.test,
# but foo.test is not needed.
-# Note that this usage has a problem: the summary will only
-# take bar.log into account, because the $(TEST_SUITE_LOG) rule
-# does not "see" baz.log. Hmm.
+# FIXME: Note that this usage has a problem: the summary will only
+# FIXME: take bar.log into account, because the $(TEST_SUITE_LOG)
+# FIXME: rule does not "see" baz.log. Hmm.
$MAKE check TESTS=bar.test >stdout && { cat stdout; Exit 1; }
cat stdout
grep '^FAIL: baz\.test$' stdout
@@ -136,7 +130,7 @@ grep '^# FAIL: *1$' stdout
grep '^# ERROR: *1$' stdout
$MAKE clean
-$MAKE check TEST_LOGS=baz.log > stdout && { cat stdout; Exit 1; }
+$MAKE check TESTS=baz > stdout && { cat stdout; Exit 1; }
cat stdout
grep foo.test stdout && Exit 1
grep bar.test stdout && Exit 1
diff --git a/t/serial-tests.sh b/t/serial-tests.sh
index 2661afa..7e406e7 100755
--- a/t/serial-tests.sh
+++ b/t/serial-tests.sh
@@ -23,7 +23,7 @@ am_create_testdir=empty
hasnt_parallel_tests ()
{
- $EGREP 'TEST_SUITE_LOG|TEST_LOGS|\.log.*:' $1 && Exit 1
+ $EGREP 'TEST_SUITE_LOG|TEST_(LOGS|BASES)|\.log.*:' $1 && Exit 1
grep 'recheck.*:' $1 && Exit 1
grep '^check-TESTS: \$(TESTS)$' $1
}
@@ -32,7 +32,7 @@ has_parallel_tests ()
{
$EGREP '(^| )check-TESTS.*:' $1
$EGREP '(^| )recheck.*:' $1
- $EGREP '^\$\(TEST_SUITE_LOG\):.* \$\(TEST_LOGS\)( |$)' $1
+ $EGREP '^\$\(TEST_SUITE_LOG\):.* \$\(am__TEST_LOGS\)( |$)' $1
grep '^%.log %.trs *:.*%\.test' $1
}
diff --git a/t/tap-basic.sh b/t/tap-basic.sh
index 417c248..a7fcd07 100755
--- a/t/tap-basic.sh
+++ b/t/tap-basic.sh
@@ -18,7 +18,7 @@
# - LOG_COMPILER support;
# - basic support for TODO and SKIP directives, and "Bail out!" magic;
# - testsuite progress output on console;
-# - runtime overriding of TESTS and TEST_LOGS;
+# - runtime overriding of TESTS;
# - correct counts of test results (both in summary and in progress
# output on console).
# Note that some of the features checked here are checked in other
@@ -144,12 +144,12 @@ grep '^PASS:' stdout && Exit 1
test `$FGREP -c ': bail.test' stdout` -eq 1
$FGREP 'success.test' stdout && Exit 1
-# Override TEST_LOGS from the command line, making it point to a test
+# Override TESTS from the command line, making it point to a test
# (ok.test) that has to be generated at make time.
rm -f *.log *.test
-$MAKE TEST_LOGS=ok.log check >stdout || { cat stdout; Exit 1; }
+$MAKE TESTS=ok check >stdout || { cat stdout; Exit 1; }
cat stdout
count_test_results total=3 pass=1 fail=0 xpass=0 xfail=1 skip=1 error=0
diff --git a/t/test-driver-custom-multitest-recheck2.sh
b/t/test-driver-custom-multitest-recheck2.sh
index eef1043..1387f12 100755
--- a/t/test-driver-custom-multitest-recheck2.sh
+++ b/t/test-driver-custom-multitest-recheck2.sh
@@ -16,8 +16,7 @@
# Custom test drivers: try the "recheck" functionality with test protocols
# that allow multiple testcases in a single test script. In particular,
-# check that this still works when we override $(TESTS) and $(TEST_LOGS)
-# at make runtime.
+# check that this still works when we override $(TESTS) at make runtime.
# See also related tests 'test-driver-custom-multitest-recheck.test' and
# 'parallel-tests-recheck-override.test'.
@@ -94,15 +93,13 @@ for vpath in : false; do
rm -f *.run
- : An empty '$(TESTS)' or '$(TEST_LOGS)' means that no test should be run.
- for var in TESTS TEST_LOGS; do
- $MAKE "$var=" recheck >stdout || { cat stdout; Exit 1; }
- cat stdout
- count_test_results total=0 pass=0 fail=0 xpass=0 xfail=0 skip=0 error=0
- test ! -r a.run
- test ! -r b.run
- test ! -r c.run
- done
+ : An empty '$(TESTS)' means that no test should be run.
+ $MAKE TESTS= recheck >stdout || { cat stdout; Exit 1; }
+ cat stdout
+ count_test_results total=0 pass=0 fail=0 xpass=0 xfail=0 skip=0 error=0
+ test ! -r a.run
+ test ! -r b.run
+ test ! -r c.run
unset var
: a.test was successful the first time, no need to re-run it.
@@ -115,7 +112,7 @@ for vpath in : false; do
: b.test failed, it should be re-run. And make it pass this time.
echo OK > b.ok
- $MAKE TEST_LOGS=b.log recheck >stdout || { cat stdout; Exit 1; }
+ $MAKE TESTS=b recheck >stdout || { cat stdout; Exit 1; }
cat stdout
test ! -r a.run
test -f b.run
@@ -125,7 +122,7 @@ for vpath in : false; do
rm -f *.run
: No need to re-run a.test or b.test anymore.
- $MAKE TEST_LOGS=b.log recheck >stdout || { cat stdout; Exit 1; }
+ $MAKE TESTS=b recheck >stdout || { cat stdout; Exit 1; }
cat stdout
count_test_results total=0 pass=0 fail=0 xpass=0 xfail=0 skip=0 error=0
test ! -r a.run
@@ -144,7 +141,7 @@ for vpath in : false; do
# Use 'echo' here, since Solaris 10 /bin/sh would try to optimize
# a ':' away after the first iteration, even if it is redirected.
echo dummy > c.err
- $MAKE TEST_LOGS='a.log c.log' recheck >stdout && { cat stdout; Exit 1; }
+ $MAKE TESTS='a.test c' recheck >stdout && { cat stdout; Exit 1; }
cat stdout
count_test_results total=1 pass=0 fail=0 xpass=0 xfail=0 skip=0 error=1
test ! -r a.run
diff --git a/t/test-missing.sh b/t/test-missing.sh
index 1547962..7ede224 100755
--- a/t/test-missing.sh
+++ b/t/test-missing.sh
@@ -14,9 +14,7 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# parallel-tests:
-# - non-existent scripts listed in TESTS get diagnosed
-# See also related test 'test-missing2.test'.
+# parallel-tests: non-existent scripts listed in TESTS get diagnosed.
am_parallel_tests=yes
. ./defs || Exit 1
@@ -50,7 +48,7 @@ cat stderr
$FGREP 'zardoz2.log' stderr
test ! -f test-suite.log
-$MAKE TEST_LOGS='zardoz3.log' check 2>stderr && { cat stderr >&2; Exit 1; }
+$MAKE TESTS='zardoz3' check 2>stderr && { cat stderr >&2; Exit 1; }
cat stderr >&2
$FGREP 'zardoz3.log' stderr
test ! -f test-suite.log
diff --git a/t/test-trs-basic.sh b/t/test-trs-basic.sh
index c988400..4b5e198 100755
--- a/t/test-trs-basic.sh
+++ b/t/test-trs-basic.sh
@@ -114,7 +114,7 @@ test -f unrelated.trs
test -f sub/foo.trs
#
-# Try with a subset of TESTS.
+# Try with a subset of TESTS, complete with test extensions.
#
$MAKE TESTS=foo.test check
@@ -135,22 +135,22 @@ test ! -f bar.trs
$MAKE clean
#
-# Try with a subset of TEST_LOGS.
+# Try with a subset of TESTS, omitting the test extensions.
#
-$MAKE TEST_LOGS=sub/zardoz.log check
+$MAKE TESTS=sub/zardoz check
test ! -f foo.trs
test ! -f bar.trs
test -f sub/zardoz.trs
$MAKE clean
test ! -f sub/zardoz.trs
-$MAKE TEST_LOGS='foo.log bar.log' check
+$MAKE TESTS='foo bar' check
test -f foo.trs
test -f bar.trs
test ! -f sub/zardoz.trs
-# "make clean" shouldn't remove '.trs' files for tests whose log
-# is not in $(TEST_LOGS).
-$MAKE TEST_LOGS=foo.log clean
+# "make clean" shouldn't remove '.trs' files for tests that are not
+# in is not in $(TESTS).
+$MAKE TESTS=foo clean
test ! -f foo.trs
test -f bar.trs
test ! -f sub/zardoz.trs
diff --git a/t/test-trs-recover.sh b/t/test-trs-recover.sh
index 8acd270..de3b771 100755
--- a/t/test-trs-recover.sh
+++ b/t/test-trs-recover.sh
@@ -102,9 +102,9 @@ test -f foo.trs
test ! -f bar.trs
test ! -f baz.trs
-: Recreate with a "make check" with redefined TEST_LOGS.
+: Recreate with a "make check" with redefined suffix-less TESTS.
rm -f foo.trs bar.trs baz.trs
-$MAKE TEST_LOGS=bar.log check
+$MAKE TESTS=bar check
test ! -f foo.trs
test -f bar.trs
test ! -f baz.trs
--
1.7.9.5
- [Automake-NG] [PATCH 00/11] Several changes to parallel-tests support, Stefano Lattarini, 2012/05/09
- [Automake-NG] [PATCH 03/11] [ng] check: refactoring to reduce code duplication, Stefano Lattarini, 2012/05/09
- [Automake-NG] [PATCH 04/11] [ng] check: refactor (better names for internal vars), Stefano Lattarini, 2012/05/09
- [Automake-NG] [PATCH 05/11] [ng] check: minor refactoring (reorder code), Stefano Lattarini, 2012/05/09
- [Automake-NG] [PATCH 02/11] [ng] refactor: get rid of am__EXEEXT (automake conditional and %transform), Stefano Lattarini, 2012/05/09
- [Automake-NG] [PATCH 01/11] [ng] check: big refactoring with semantic changes in parallel-tests support,
Stefano Lattarini <=
- [Automake-NG] [PATCH 08/11] [ng] check: AM_LAZY_CHECK="yes", not RECHECK_TESTS="", for lazy re-runs, Stefano Lattarini, 2012/05/09
- [Automake-NG] [PATCH 07/11] [ng] check: refactor: split recipes for check-TESTS and recheck, Stefano Lattarini, 2012/05/09
- [Automake-NG] [PATCH 06/11] [ng] check: minor refactoring (prefer make time over recipe time), Stefano Lattarini, 2012/05/09
- [Automake-NG] [PATCH 11/11] [ng] check: per-suffix dependencies for test cases, Stefano Lattarini, 2012/05/09
- [Automake-NG] [PATCH 10/11] [ng] check: few minor simplifications, Stefano Lattarini, 2012/05/09
- [Automake-NG] [PATCH 09/11] [ng] tests: remove overly picky test cases (and fix few typos), Stefano Lattarini, 2012/05/09
- Re: [Automake-NG] [PATCH 00/11] Several changes to parallel-tests support, Akim Demaille, 2012/05/09