[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Automake-commit] [SCM] GNU Automake branch, test-protocols, updated. v1
|
From: |
Stefano Lattarini |
|
Subject: |
[Automake-commit] [SCM] GNU Automake branch, test-protocols, updated. v1.11-1061-ge698ee3 |
|
Date: |
Thu, 25 Aug 2011 12:03:21 +0000 |
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "GNU Automake".
http://git.sv.gnu.org/gitweb/?p=automake.git;a=commitdiff;h=e698ee3ea801e1a02263516d75bc4fb1f25e7b3c
The branch, test-protocols has been updated
via e698ee3ea801e1a02263516d75bc4fb1f25e7b3c (commit)
via a5b9cae78ce16229fd1cdc7459b08be60bdd6c72 (commit)
via 33d456f2f6387bc2ead8fbe16d67a77edc0336eb (commit)
via aba3010e83194e2172006059db08f4fd5992f021 (commit)
via 3cb4cb584d97f56feb7247e9decec70a7d88212e (commit)
via 4f394b7451ee7defec36e3c3d845d0f932c20cf6 (commit)
via a822d925d597c5c0f67e3c9ad9642cfcf70ec7fc (commit)
from 30913e0eb8ef1f9924b6762d277a561c8de9e622 (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
commit e698ee3ea801e1a02263516d75bc4fb1f25e7b3c
Author: Stefano Lattarini <address@hidden>
Date: Thu Aug 25 13:22:58 2011 +0200
tap/awk: allow escaping of TAP directives
* lib/tap-driver.sh (setup_result_obj): Handle escaping of TAP
directives in a way tat is (mostly) compatible by what is done
by the TAP::Parser module.
With this change, the tests `tap-escape-directive.test' and
`tap-escape-directive-2.test' now also pass with the shell/awk
implementation of the TAP driver.
commit a5b9cae78ce16229fd1cdc7459b08be60bdd6c72
Author: Stefano Lattarini <address@hidden>
Date: Thu Aug 25 12:55:55 2011 +0200
coverage: more about escaping of TAP directives
* tests/tap-escape-directive.test: Make grepping of testsuite
progress output slightly stricter.
* tests/tap-escape-directive-2.test: New test (failing with the
awk implementation of the TAP driver).
* tests/Makefile.am (tap_with_common_setup_tests): Update.
commit 33d456f2f6387bc2ead8fbe16d67a77edc0336eb
Author: Stefano Lattarini <address@hidden>
Date: Thu Aug 25 12:44:32 2011 +0200
tap: improve syncing between awk+shell and perl implementations
* lib/tap-driver.pl (stringify_test_result): Renamed ...
(stringify_result_obj): ... to this. Break up a clause in the
long "if/elsif/.../else" construct to avoid unaesthetic line
breaks and to be more synced with the sibling function in
`tap-driver.sh'. Rename the `$result', `$PASS' and `$FAIL'
variables to respectively `$result_obj', `$COOKED_PASS' and
`$COOKED_FAIL', for clarity and better syncing.
(handle_tap_test): Renamed ...
(handle_tap_result): ... to this, and change the name of the
`$test' local variable to `$result_obj'.
(extract_comment): Reimplement using the simpler `index' and
`substr' builtins, rather than with more advanced uses of
regular expressions.
(%test_results, @test_results): Renamed respectively ...
(%test_results_seen, @test_results_list): ... to these, and
related adjustments throughout the `TEST_RESULTS' block.
(main, get_global_test_result): Refactor and do some cosmetic
changes to make these functions clearer and better synced with
sibling code in `tap-driver.sh'.
Other minor cosmetic and typo fixes.
* lib/tap-driver.sh (extract_tap_comment): Remove outdated
"FIXME" comments.
(get_global_test_result): Small reordering to make it better
synced with its sibling function in `tap-driver.pl'.
(stringify_result_obj): Consistently use `result_obj' as the
parameter name.
Other minor cosmetic and typo fixes.
commit aba3010e83194e2172006059db08f4fd5992f021
Author: Stefano Lattarini <address@hidden>
Date: Thu Aug 25 10:53:23 2011 +0200
tap/perl: don't redirect perl warnings/errors to log files
With this change, the test `tap-driver-stderr.test' also passes
with the perl implementation of the TAP driver.
* lib/tap-driver.pl (start): Save the original stderr into the
`OLDERR' file handle, and call ...
(trap_perl_warnings_and_errors): ... this new function, trapping
the `__WARN__' and `__DIE__' "pseudo-signals" to ensure that the
warning and error messages goes to that original stderr stream.
Since we are at it, be sure to prepend all possible "die" message
with the name of the script as given by the `$ME' variable.
commit 3cb4cb584d97f56feb7247e9decec70a7d88212e
Author: Stefano Lattarini <address@hidden>
Date: Thu Aug 25 10:05:51 2011 +0200
tap/perl: add copyright notice, version string, and emacs stuff
* lib/tap-driver.pl: Add proper copyright notice and bug reporting
address. Remove obsolete heading comments. Add trailing comments
for emacs w.r.t. code-formatting rules and automatic updating of
the date-based script version saved into the `$VERSION' variable.
($VERSION): Define to the current UTC date.
commit 4f394b7451ee7defec36e3c3d845d0f932c20cf6
Author: Stefano Lattarini <address@hidden>
Date: Wed Aug 24 11:27:00 2011 +0200
tap/awk: don't redirect awk stderr to log files
* lib/tap-driver.sh: Handle the file descriptors redirections
more smartly, to avoid sending error messages from awk (about
e.g. syntax or I/O errors) to the log files instead that to the
console.
* tests/tap-driver-stderr.test: New test, verifying the improved
driver behaviour. Notice that this test still fails when using
the perl implementation of the TAP driver.
* tests/Makefile.am (tap_other_tests): Update.
commit a822d925d597c5c0f67e3c9ad9642cfcf70ec7fc
Author: Stefano Lattarini <address@hidden>
Date: Wed Aug 24 09:54:22 2011 +0200
tap/awk: remove obsolete "FIXME" comment
* lib/tap-driver.sh: Remove obsolete "FIXME" comment about our
driver losing the exit status of the tested program; this issue
has been solved in precedent commit `v1.11-1052-gd630a0d'. Fix
an unrelated typo in comments since we are at it.
-----------------------------------------------------------------------
Summary of changes:
ChangeLog | 92 ++++++++++
lib/tap-driver.pl | 188 +++++++++++++-------
lib/tap-driver.sh | 62 ++++---
tests/Makefile.am | 2 +
tests/Makefile.in | 2 +
...ap-common-setup.test => tap-driver-stderr.test} | 38 +++--
...-directive.test => tap-escape-directive-2.test} | 28 +++-
tests/tap-escape-directive.test | 4 +-
8 files changed, 304 insertions(+), 112 deletions(-)
copy tests/{tap-common-setup.test => tap-driver-stderr.test} (56%)
copy tests/{tap-escape-directive.test => tap-escape-directive-2.test} (64%)
diff --git a/ChangeLog b/ChangeLog
index 7a161ac..e407d89 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,95 @@
+2011-08-25 Stefano Lattarini <address@hidden>
+
+ tap/awk: allow escaping of TAP directives
+ * lib/tap-driver.sh (setup_result_obj): Handle escaping of TAP
+ directives in a way tat is (mostly) compatible by what is done
+ by the TAP::Parser module.
+ With this change, the tests `tap-escape-directive.test' and
+ `tap-escape-directive-2.test' now also pass with the shell/awk
+ implementation of the TAP driver.
+
+2011-08-25 Stefano Lattarini <address@hidden>
+
+ coverage: more about escaping of TAP directives
+ * tests/tap-escape-directive.test: Make grepping of testsuite
+ progress output slightly stricter.
+ * tests/tap-escape-directive-2.test: New test (failing with the
+ awk implementation of the TAP driver).
+ * tests/Makefile.am (tap_with_common_setup_tests): Update.
+
+2011-08-25 Stefano Lattarini <address@hidden>
+
+ tap: improve syncing between awk+shell and perl implementations
+ * lib/tap-driver.pl (stringify_test_result): Renamed ...
+ (stringify_result_obj): ... to this. Break up a clause in the
+ long "if/elsif/.../else" construct to avoid unaesthetic line
+ breaks and to be more synced with the sibling function in
+ `tap-driver.sh'. Rename the `$result', `$PASS' and `$FAIL'
+ variables to respectively `$result_obj', `$COOKED_PASS' and
+ `$COOKED_FAIL', for clarity and better syncing.
+ (handle_tap_test): Renamed ...
+ (handle_tap_result): ... to this, and change the name of the
+ `$test' local variable to `$result_obj'.
+ (extract_comment): Reimplement using the simpler `index' and
+ `substr' builtins, rather than with more advanced uses of
+ regular expressions.
+ (%test_results, @test_results): Renamed respectively ...
+ (%test_results_seen, @test_results_list): ... to these, and
+ related adjustments throughout the `TEST_RESULTS' block.
+ (main, get_global_test_result): Refactor and do some cosmetic
+ changes to make these functions clearer and better synced with
+ sibling code in `tap-driver.sh'.
+ Other minor cosmetic and typo fixes.
+ * lib/tap-driver.sh (extract_tap_comment): Remove outdated
+ "FIXME" comments.
+ (get_global_test_result): Small reordering to make it better
+ synced with its sibling function in `tap-driver.pl'.
+ (stringify_result_obj): Consistently use `result_obj' as the
+ parameter name.
+ Other minor cosmetic and typo fixes.
+
+2011-08-25 Stefano Lattarini <address@hidden>
+
+ tap/perl: don't redirect perl warnings/errors to log files
+ With this change, the test `tap-driver-stderr.test' also passes
+ with the perl implementation of the TAP driver.
+ * lib/tap-driver.pl (start): Save the original stderr into the
+ `OLDERR' file handle, and call ...
+ (trap_perl_warnings_and_errors): ... this new function, trapping
+ the `__WARN__' and `__DIE__' "pseudo-signals" to ensure that the
+ warning and error messages goes to that original stderr stream.
+ Since we are at it, be sure to prepend all possible "die" message
+ with the name of the script as given by the `$ME' variable.
+
+2011-08-25 Stefano Lattarini <address@hidden>
+
+ tap/perl: add copyright notice, version string, and emacs stuff
+ * lib/tap-driver.pl: Add proper copyright notice and bug reporting
+ address. Remove obsolete heading comments. Add trailing comments
+ for emacs w.r.t. code-formatting rules and automatic updating of
+ the date-based script version saved into the `$VERSION' variable.
+ ($VERSION): Define to the current UTC date.
+
+2011-08-24 Stefano Lattarini <address@hidden>
+
+ tap/awk: don't redirect awk stderr to log files
+ * lib/tap-driver.sh: Handle the file descriptors redirections
+ more smartly, to avoid sending error messages from awk (about
+ e.g. syntax or I/O errors) to the log files instead that to the
+ console.
+ * tests/tap-driver-stderr.test: New test, verifying the improved
+ driver behaviour. Notice that this test still fails when using
+ the perl implementation of the TAP driver.
+ * tests/Makefile.am (tap_other_tests): Update.
+
+2011-08-24 Stefano Lattarini <address@hidden>
+
+ tap/awk: remove obsolete "FIXME" comment
+ * lib/tap-driver.sh: Remove obsolete "FIXME" comment about our
+ driver losing the exit status of the tested program; this issue
+ has been solved in precedent commit `v1.11-1052-gd630a0d'. Fix
+ an unrelated typo in comments since we are at it.
+
2011-08-22 Stefano Lattarini <address@hidden>
testsuite: avoid spurious failure on Solaris
diff --git a/lib/tap-driver.pl b/lib/tap-driver.pl
index 9e1ece4..2c328d9 100755
--- a/lib/tap-driver.pl
+++ b/lib/tap-driver.pl
@@ -1,6 +1,27 @@
#! /usr/bin/env perl
-# Temporary/experimental TAP test driver for Automake.
-# TODO: should be rewritten portably (e.g., in awk or shell).
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# As a special exception to the GNU General Public License, if you
+# distribute this file as part of a program that contains a
+# configuration script generated by Autoconf, you may include it under
+# the same distribution terms that you use for the rest of that program.
+
+# This file is maintained in Automake, please report
+# bugs to <address@hidden> or send patches to
+# <address@hidden>.
# ---------------------------------- #
# Imports, static data, and setup. #
@@ -11,6 +32,8 @@ use strict;
use Getopt::Long ();
use TAP::Parser;
+my $VERSION = '2011-08-25.10'; # UTC
+
my $ME = "tap-driver.pl";
my $USAGE = <<'END';
@@ -26,8 +49,6 @@ END
my $HELP = "$ME: TAP-aware test driver for Automake testsuite harness." .
"\n" . $USAGE;
-my $VERSION = '(experimental version)';
-
# Keep this in sync with `lib/am/check.am:$(am__tty_colors)'.
my %COLOR = (
red => "\e[0;31m",
@@ -106,14 +127,15 @@ sub get_test_exit_message ();
sub get_test_results ();
sub handle_tap_bailout ($);
sub handle_tap_plan ($);
-sub handle_tap_test ($);
+sub handle_tap_result ($);
sub is_null_string ($);
sub main (@);
sub must_recheck ();
sub report ($;$);
sub start (@);
-sub stringify_test_result ($);
+sub stringify_result_obj ($);
sub testsuite_error ($);
+sub trap_perl_warnings_and_errors ();
sub write_test_results ();
sub yn ($);
@@ -134,7 +156,7 @@ sub bool_opt ($$)
}
else
{
- die "invalid argument '$val' for option '$opt'\n";
+ die "$ME: invalid argument '$val' for option '$opt'\n";
}
}
@@ -157,41 +179,42 @@ sub yn ($)
TEST_RESULTS :
{
- my (@test_results, %test_results);
+ my (@test_results_list, %test_results_seen);
sub add_test_result ($)
{
my $res = shift;
- push @test_results, $res;
- $test_results{$res} = 1;
+ push @test_results_list, $res;
+ $test_results_seen{$res} = 1;
}
sub get_test_results ()
{
- return @test_results;
+ return @test_results_list;
}
# Whether the test script should be re-run by "make recheck".
sub must_recheck ()
{
- return grep { !/^(?:XFAIL|PASS|SKIP)$/ } (keys %test_results);
+ return grep { !/^(?:XFAIL|PASS|SKIP)$/ } (keys %test_results_seen);
}
# Whether the content of the log file associated to this test should
# be copied into the "global" test-suite.log.
sub copy_in_global_log ()
{
- return grep { not $_ eq "PASS" } (keys %test_results);
+ return grep { not $_ eq "PASS" } (keys %test_results_seen);
}
# FIXME: this can certainly be improved ...
sub get_global_test_result ()
{
- my @results = keys %test_results;
- return "ERROR" if exists $test_results{"ERROR"};
- return "SKIP" if @results == 1 && $results[0] eq "SKIP";
- return "FAIL" if exists $test_results{"FAIL"};
- return "FAIL" if exists $test_results{"XPASS"};
+ return "ERROR"
+ if $test_results_seen{"ERROR"};
+ return "FAIL"
+ if $test_results_seen{"FAIL"} || $test_results_seen{"XPASS"};
+ return "SKIP"
+ if scalar keys %test_results_seen == 1 && $test_results_seen{"SKIP"};
return "PASS";
}
@@ -199,7 +222,7 @@ TEST_RESULTS :
sub write_test_results ()
{
- open RES, ">", $trs_file or die "opening $trs_file: $!\n";
+ open RES, ">", $trs_file or die "$ME: opening $trs_file: $!\n";
print RES ":global-test-result: " . get_global_test_result . "\n";
print RES ":recheck: " . yn (must_recheck) . "\n";
print RES ":copy-in-global-log: " . yn (copy_in_global_log) . "\n";
@@ -207,18 +230,32 @@ sub write_test_results ()
{
print RES ":test-result: $result\n";
}
- close RES or die "closing $trs_file: $!\n";
+ close RES or die "$ME: closing $trs_file: $!\n";
+}
+
+sub trap_perl_warnings_and_errors ()
+{
+ $SIG{__WARN__} = $SIG{__DIE__} = sub
+ {
+ # Be sure to send the warning/error message to the original stderr
+ # (presumably the console), not into the log file.
+ open STDERR, ">&", \*OLDERR;
+ die @_;
+ }
}
sub start (@)
{
# Redirect stderr and stdout to a temporary log file. Save the
# original stdout stream, since we need it to print testsuite
- # progress output.
- open LOG, ">", $log_file or die "opening $log_file: $!\n";
- open OLDOUT, ">&STDOUT" or die "duplicating stdout: $!\n";
- open STDOUT, ">&LOG" or die "redirecting stdout: $!\n";
- open STDERR, ">&LOG" or die "redirecting stderr: $!\n";
+ # progress output. Save original stderr stream, so that we can
+ # redirect warning and error messages from perl there.
+ open LOG, ">", $log_file or die "$ME: opening $log_file: $!\n";
+ open OLDOUT, ">&STDOUT" or die "$ME: duplicating stdout: $!\n";
+ open OLDERR, ">&STDERR" or die "$ME: duplicating stdout: $!\n";
+ trap_perl_warnings_and_errors;
+ open STDOUT, ">&LOG" or die "$ME: redirecting stdout: $!\n";
+ open STDERR, ">&LOG" or die "$ME: redirecting stderr: $!\n";
$parser = TAP::Parser->new ({ exec => address@hidden, merge => $cfg{merge}
});
$parser->ignore_exit(1) if $cfg{"ignore-exit"};
}
@@ -227,7 +264,7 @@ sub get_test_exit_message ()
{
my $wstatus = $parser->wait;
# Watch out for possible internal errors.
- die "couldn't get the exit ststus of the TAP producer"
+ die "$ME: couldn't get the exit ststus of the TAP producer"
unless defined $wstatus;
# Return an undefined value if the producer exited with success.
return unless $wstatus;
@@ -248,36 +285,38 @@ sub get_test_exit_message ()
}
}
-sub stringify_test_result ($)
+sub stringify_result_obj ($)
{
- my $result = shift;
- my $PASS = $cfg{"expect-failure"} ? "XPASS": "PASS";
- my $FAIL = $cfg{"expect-failure"} ? "XFAIL": "FAIL";
- if ($result->is_unplanned
- || $result->number != $testno
- || $plan_seen == LATE_PLAN)
+ my $result_obj = shift;
+ my $COOKED_PASS = $cfg{"expect-failure"} ? "XPASS": "PASS";
+ my $COOKED_FAIL = $cfg{"expect-failure"} ? "XFAIL": "FAIL";
+ if ($result_obj->is_unplanned || $result_obj->number != $testno)
{
return "ERROR";
}
- elsif (!$result->directive)
+ elsif ($plan_seen == LATE_PLAN)
{
- return $result->is_ok ? $PASS: $FAIL;
+ return "ERROR";
+ }
+ elsif (!$result_obj->directive)
+ {
+ return $result_obj->is_ok ? $COOKED_PASS: $COOKED_FAIL;
}
- elsif ($result->has_todo)
+ elsif ($result_obj->has_todo)
{
- return $result->is_actual_ok ? "XPASS" : "XFAIL";
+ return $result_obj->is_actual_ok ? "XPASS" : "XFAIL";
}
- elsif ($result->has_skip)
+ elsif ($result_obj->has_skip)
{
- return $result->is_ok ? "SKIP" : $FAIL;
+ return $result_obj->is_ok ? "SKIP" : $COOKED_FAIL;
}
- die "INTERNAL ERROR"; # NOTREACHED
+ die "$ME: INTERNAL ERROR"; # NOTREACHED
}
sub colored ($$)
{
my ($color_name, $text) = @_;
- return $COLOR{$color_name} . $text . $COLOR{'std'};
+ return $COLOR{$color_name} . $text . $COLOR{'std'};
}
sub decorate_result ($)
@@ -317,7 +356,7 @@ sub report ($;$)
}
else
{
- die "INTERNAL ERROR"; # NOTREACHED
+ die "$ME: INTERNAL ERROR"; # NOTREACHED
}
$msg .= " $explanation" if defined $explanation;
$msg .= "\n";
@@ -333,15 +372,15 @@ sub testsuite_error ($)
report "ERROR", "- $_[0]";
}
-sub handle_tap_test ($)
+sub handle_tap_result ($)
{
$testno++;
- my $test = shift;
+ my $result_obj = shift;
- my $test_result = stringify_test_result $test;
- my $string = $test->number;
+ my $test_result = stringify_result_obj $result_obj;
+ my $string = $result_obj->number;
- my $description = $test->description;
+ my $description = $result_obj->description;
$string .= " $description"
unless is_null_string $description;
@@ -349,18 +388,18 @@ sub handle_tap_test ($)
{
$string .= " # AFTER LATE PLAN";
}
- elsif ($test->is_unplanned)
+ elsif ($result_obj->is_unplanned)
{
$string .= " # UNPLANNED";
}
- elsif ($test->number != $testno)
+ elsif ($result_obj->number != $testno)
{
$string .= " # OUT-OF-ORDER (expecting $testno)";
}
- elsif (my $directive = $test->directive)
+ elsif (my $directive = $result_obj->directive)
{
$string .= " # $directive";
- my $explanation = $test->explanation;
+ my $explanation = $result_obj->explanation;
$string .= " $explanation"
unless is_null_string $explanation;
}
@@ -408,11 +447,15 @@ sub handle_tap_bailout ($)
sub extract_tap_comment ($)
{
- local $_ = shift;
- if (/^\Q$diag_string\E(.*)$/o)
+ my $line = shift;
+ if (index ($line, $diag_string) == 0)
{
- (my $comment = $1) =~ s/(?:^\s*|\s*$)//g;
- return $comment;
+ # Strip leading `$diag_string' from `$line'.
+ $line = substr ($line, length ($diag_string));
+ # And strip any leading and trailing whitespace left.
+ $line =~ s/(?:^\s*|\s*$)//g;
+ # Return what is left (if any).
+ return $line;
}
return "";
}
@@ -434,7 +477,7 @@ sub main (@)
}
elsif ($cur->is_test)
{
- handle_tap_test ($cur);
+ handle_tap_result ($cur);
}
elsif ($cur->is_bailout)
{
@@ -461,14 +504,14 @@ sub main (@)
testsuite_error (sprintf "too %s tests run (expected %d, got %d)",
$bad_amount, $planned, $run);
}
- }
- if (!$cfg{"ignore-exit"} && !$bailed_out)
- {
- my $msg = get_test_exit_message ();
- testsuite_error $msg if $msg;
+ if (!$cfg{"ignore-exit"})
+ {
+ my $msg = get_test_exit_message ();
+ testsuite_error $msg if $msg;
+ }
}
write_test_results;
- close LOG or die "closing $log_file: $!\n";
+ close LOG or die "$ME: closing $log_file: $!\n";
exit 0;
}
@@ -478,4 +521,23 @@ sub main (@)
main @ARGV;
-# vim: ft=perl ts=4 sw=4 et
+# Local Variables:
+# perl-indent-level: 2
+# perl-continued-statement-offset: 2
+# perl-continued-brace-offset: 0
+# perl-brace-offset: 0
+# perl-brace-imaginary-offset: 0
+# perl-label-offset: -2
+# cperl-indent-level: 2
+# cperl-brace-offset: 0
+# cperl-continued-brace-offset: 0
+# cperl-label-offset: -2
+# cperl-extra-newline-before-brace: t
+# cperl-merge-trailing-else: nil
+# cperl-continued-statement-offset: 2
+# eval: (add-hook 'write-file-hooks 'time-stamp)
+# time-stamp-start: "my $VERSION = "
+# time-stamp-format: "'%:y-%02m-%02d.%02H'"
+# time-stamp-time-zone: "UTC"
+# time-stamp-end: "; # UTC"
+# End:
diff --git a/lib/tap-driver.sh b/lib/tap-driver.sh
index 1255662..44317d9 100755
--- a/lib/tap-driver.sh
+++ b/lib/tap-driver.sh
@@ -23,7 +23,7 @@
# bugs to <address@hidden> or send patches to
# <address@hidden>.
-scriptversion=2011-08-21.21; # UTC
+scriptversion=2011-08-25.11; # UTC
# Make unconditional expansion of undefined variables an error. This
# helps a lot in preventing typo-related bugs.
@@ -116,13 +116,14 @@ else
fi
{
- # FIXME: this usage loses the test program exit status. We should
- # probably rewrite the awk script to use the
- # expression | getline [var]
- # idiom, which should allow us to obtain the final exit status from
- # <expression> when closing it.
- { test $merge -eq 0 || exec 2>&1; "$@"; echo $?; } \
- | LC_ALL=C ${AM_TAP_AWK-awk} \
+ { if test $merge -gt 0; then
+ exec 2>&1
+ else
+ exec 2>&3
+ fi
+ "$@"
+ echo $?
+ } | LC_ALL=C ${AM_TAP_AWK-awk} \
-v me="$me" \
-v test_script_name="$test_name" \
-v log_file="$log_file" \
@@ -133,7 +134,7 @@ fi
-v comments="$comments" \
-v diag_string="$diag_string" \
'
-# FIXME: the usages of "cat >&3" below could be optimized whne using
+# FIXME: the usages of "cat >&3" below could be optimized when using
# FIXME: GNU awk, and/on on systems that supports /dev/fd/.
# Implementation note: in what follows, `result_obj` will be an
@@ -146,7 +147,7 @@ fi
function fatal(msg)
{
- print me ": " msg | "cat >&3"
+ print me ": " msg | "cat >&2"
exit 1
}
@@ -194,35 +195,35 @@ function get_global_test_result()
{
if ("ERROR" in test_results_seen)
return "ERROR"
+ if ("FAIL" in test_results_seen || "XPASS" in test_results_seen)
+ return "FAIL"
all_skipped = 1
for (k in test_results_seen)
if (k != "SKIP")
all_skipped = 0
if (all_skipped)
return "SKIP"
- if ("FAIL" in test_results_seen || "XPASS" in test_results_seen)
- return "FAIL"
return "PASS";
}
-function stringify_result_obj(obj)
+function stringify_result_obj(result_obj)
{
- if (obj["is_unplanned"] || obj["number"] != testno)
+ if (result_obj["is_unplanned"] || result_obj["number"] != testno)
return "ERROR"
if (plan_seen == LATE_PLAN)
return "ERROR"
if (result_obj["directive"] == "TODO")
- return obj["is_ok"] ? "XPASS" : "XFAIL"
+ return result_obj["is_ok"] ? "XPASS" : "XFAIL"
if (result_obj["directive"] == "SKIP")
- return obj["is_ok"] ? "SKIP" : COOKED_FAIL;
+ return result_obj["is_ok"] ? "SKIP" : COOKED_FAIL;
if (length(result_obj["directive"]))
abort("in function stringify_result_obj()")
- return obj["is_ok"] ? COOKED_PASS : COOKED_FAIL
+ return result_obj["is_ok"] ? COOKED_PASS : COOKED_FAIL
}
function decorate_result(result)
@@ -253,10 +254,10 @@ function report(result, details)
if (length(details))
msg = msg " " details
# Output on console might be colorized.
- print decorate_result(result) msg | "cat >&3";
+ print decorate_result(result) msg
# Log the result in the log file too, to help debugging (this is
# especially true when said result is a TAP error or "Bail out!").
- print result msg;
+ print result msg | "cat >&3";
}
function testsuite_error(error_message)
@@ -293,7 +294,7 @@ function handle_tap_result()
report(stringify_result_obj(result_obj), details)
}
-# `skip_reason` should be emprty whenever planned > 0.
+# `skip_reason` should be empty whenever planned > 0.
function handle_tap_plan(planned, skip_reason)
{
planned += 0 # Avoid getting confused if, say, `planned` is "00"
@@ -328,11 +329,9 @@ function handle_tap_plan(planned, skip_reason)
function extract_tap_comment(line)
{
- # FIXME: verify there is not an off-by-one bug here.
if (index(line, diag_string) == 1)
{
# Strip leading `diag_string` from `line`.
- # FIXME: verify there is not an off-by-one bug here.
line = substr(line, length(diag_string) + 1)
# And strip any leading and trailing whitespace left.
sub("^[ \t]*", "", line)
@@ -383,7 +382,6 @@ function setup_result_obj(line)
result_obj["directive"] = ""
result_obj["explanation"] = ""
- # TODO: maybe we should allow a way to escape "#"?
if (index(line, "#") == 0)
return # No possible directive, nothing more to do.
@@ -399,6 +397,20 @@ function setup_result_obj(line)
if (!pos)
return
+ # Let`s now see if the TAP directive has been escaped. For example:
+ # escaped: ok \# SKIP
+ # not escaped: ok \\# SKIP
+ # escaped: ok \\\\\# SKIP
+ # not escaped: ok \ # SKIP
+ if (substr(line, pos, 1) == "#")
+ {
+ bslash_count = 0
+ for (i = pos; i > 1 && substr(line, i - 1, 1) == "\\"; i--)
+ bslash_count += 1
+ if (bslash_count % 2)
+ return # Directive was escaped.
+ }
+
# Strip the directive and its explanation (if any) from the test
# description.
result_obj["description"] = substr(line, 1, pos - 1)
@@ -503,7 +515,7 @@ while (1)
$0 = curline
}
# Copy any input line verbatim into the log file.
- print
+ print | "cat >&3"
# Parsing of TAP input should stop after a "Bail out!" directive.
if (bailed_out)
continue
@@ -593,7 +605,7 @@ exit 0
'
# TODO: document that we consume the file descriptor 3 :-(
-} 3>&1 >"$log_file" 2>&1
+} 3>"$log_file"
test $? -eq 0 || fatal "I/O or internal error"
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 4c8f342..bf0e07d 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -1155,6 +1155,7 @@ tap-diagnostic.test \
tap-empty-diagnostic.test \
tap-empty.test \
tap-escape-directive.test \
+tap-escape-directive-2.test \
tap-exit.test \
tap-signal.test \
tap-fancy.test \
@@ -1225,6 +1226,7 @@ tap-common-setup.test \
tap-bad-prog.tap \
tap-basic.test \
tap-diagnostic-custom.test \
+tap-driver-stderr.test \
tap-doc.test \
tap-doc2.test \
tap-more.test \
diff --git a/tests/Makefile.in b/tests/Makefile.in
index d9fafa7..b888c3b 100644
--- a/tests/Makefile.in
+++ b/tests/Makefile.in
@@ -1396,6 +1396,7 @@ tap-diagnostic.test \
tap-empty-diagnostic.test \
tap-empty.test \
tap-escape-directive.test \
+tap-escape-directive-2.test \
tap-exit.test \
tap-signal.test \
tap-fancy.test \
@@ -1463,6 +1464,7 @@ tap-common-setup.test \
tap-bad-prog.tap \
tap-basic.test \
tap-diagnostic-custom.test \
+tap-driver-stderr.test \
tap-doc.test \
tap-doc2.test \
tap-more.test \
diff --git a/tests/tap-common-setup.test b/tests/tap-driver-stderr.test
similarity index 56%
copy from tests/tap-common-setup.test
copy to tests/tap-driver-stderr.test
index 92ae3c4..396624e 100755
--- a/tests/tap-common-setup.test
+++ b/tests/tap-driver-stderr.test
@@ -14,30 +14,36 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Auxiliary test to set up common data used by many tests on TAP support.
+# Unit test on TAP driver:
+# - error messages from awk/shell/perl goes to the console
+required=non-root
parallel_tests=yes
. ./defs || Exit 1
-cat >> configure.in << END
-AC_OUTPUT
-END
+fetch_tap_driver
-cat > Makefile.am << 'END'
-TEST_LOG_DRIVER = $(srcdir)/tap-driver
-TEST_LOG_COMPILER = cat
-TESTS = all.test
-END
+tst=zardoz
-$ACLOCAL
-$AUTOCONF
-$AUTOMAKE
+for suf in trs log; do
-./configure
+ rm -f $tst.log $tst.trs
+ touch $tst.$suf
+ chmod a-w $tst.$suf
-rm -rf autom4te*.cache
+ st=0
+ ./tap-driver --test-name $tst --log-file $tst.log --trs-file $tst.trs \
+ -- sh -c 'echo 1..1; echo ok 1; echo "Hello, World!"' \
+ >stdout 2>stderr && st=1
+ cat stdout
+ cat stderr >&2
+ cat $tst.log || :
+ cat $tst.trs || :
+ test $st -eq 0
-# So that the data files we've created won't be removed at exit.
-keep_testdirs=yes
+ $FGREP 'Hello, World!' stderr stdout && Exit 1
+ $FGREP $tst.$suf stderr
+
+done
:
diff --git a/tests/tap-escape-directive.test b/tests/tap-escape-directive-2.test
similarity index 64%
copy from tests/tap-escape-directive.test
copy to tests/tap-escape-directive-2.test
index dc03197..e0dd37b 100755
--- a/tests/tap-escape-directive.test
+++ b/tests/tap-escape-directive-2.test
@@ -23,17 +23,33 @@ parallel_tests=yes
. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
cat > all.test <<'END'
-1..2
-ok \# TODO
-ok \# SKIP
+1..8
+
+not ok \ # TODO
+ok \ # SKIP
+
+not ok \\# TODO
+ok \\# SKIP
+
+ok \\\# TODO
+ok \\\# SKIP
+
+not ok \\\\\\\\\\# TODO
+ok \\\\\\\\\\# SKIP
END
$MAKE check >stdout || { cat stdout; Exit 1; }
cat stdout
-count_test_results total=2 pass=2 fail=0 xpass=0 xfail=0 skip=0 error=0
+count_test_results total=8 pass=2 fail=0 xpass=0 xfail=3 skip=3 error=0
-grep '^PASS: all\.test 1 .*TODO' stdout
-grep '^PASS: all\.test 2 .*SKIP' stdout
+grep '^XFAIL: all\.test 1 .*# TODO' stdout
+grep '^SKIP: all\.test 2 .*# SKIP' stdout
+grep '^XFAIL: all\.test 3 .*# TODO' stdout
+grep '^SKIP: all\.test 4 .*# SKIP' stdout
+grep '^PASS: all\.test 5 .*# TODO' stdout
+grep '^PASS: all\.test 6 .*# SKIP' stdout
+grep '^XFAIL: all\.test 7 .*# TODO' stdout
+grep '^SKIP: all\.test 8 .*# SKIP' stdout
:
diff --git a/tests/tap-escape-directive.test b/tests/tap-escape-directive.test
index dc03197..194db6a 100755
--- a/tests/tap-escape-directive.test
+++ b/tests/tap-escape-directive.test
@@ -33,7 +33,7 @@ cat stdout
count_test_results total=2 pass=2 fail=0 xpass=0 xfail=0 skip=0 error=0
-grep '^PASS: all\.test 1 .*TODO' stdout
-grep '^PASS: all\.test 2 .*SKIP' stdout
+grep '^PASS: all\.test 1 .*# TODO' stdout
+grep '^PASS: all\.test 2 .*# SKIP' stdout
:
hooks/post-receive
--
GNU Automake
| [Prev in Thread] |
Current Thread |
[Next in Thread] |
- [Automake-commit] [SCM] GNU Automake branch, test-protocols, updated. v1.11-1061-ge698ee3,
Stefano Lattarini <=