[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Automake-commit] [SCM] GNU Automake branch, testsuite-work, updated. v1
|
From: |
Stefano Lattarini |
|
Subject: |
[Automake-commit] [SCM] GNU Automake branch, testsuite-work, updated. v1.11-1247-g449a81c |
|
Date: |
Fri, 26 Aug 2011 09:23:32 +0000 |
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "GNU Automake".
http://git.sv.gnu.org/gitweb/?p=automake.git;a=commitdiff;h=449a81c2fd090be94d6128aba1dfbcfd9e15e4ae
The branch, testsuite-work has been updated
via 449a81c2fd090be94d6128aba1dfbcfd9e15e4ae (commit)
via 85aa4fee2425be149d54a634f7664d70184c5b41 (commit)
via e698ee3ea801e1a02263516d75bc4fb1f25e7b3c (commit)
via a5b9cae78ce16229fd1cdc7459b08be60bdd6c72 (commit)
via 33d456f2f6387bc2ead8fbe16d67a77edc0336eb (commit)
via aba3010e83194e2172006059db08f4fd5992f021 (commit)
via 3cb4cb584d97f56feb7247e9decec70a7d88212e (commit)
via 4f394b7451ee7defec36e3c3d845d0f932c20cf6 (commit)
via a822d925d597c5c0f67e3c9ad9642cfcf70ec7fc (commit)
from 451ab178e635edec7c320034b47bcd96c770d1a4 (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
commit 449a81c2fd090be94d6128aba1dfbcfd9e15e4ae
Merge: 451ab17 85aa4fe
Author: Stefano Lattarini <address@hidden>
Date: Fri Aug 26 11:16:41 2011 +0200
Merge branch 'test-protocols' into testsuite-work
* test-protocols:
tests: fix spurious failure for awk implementation of TAP driver
tap/awk: allow escaping of TAP directives
coverage: more about escaping of TAP directives
tap: improve syncing between awk+shell and perl implementations
tap/perl: don't redirect perl warnings/errors to log files
tap/perl: add copyright notice, version string, and emacs stuff
tap/awk: don't redirect awk stderr to log files
tap/awk: remove obsolete "FIXME" comment
-----------------------------------------------------------------------
Summary of changes:
ChangeLog | 101 +++++++++++
lib/tap-driver.pl | 188 +++++++++++++-------
lib/tap-driver.sh | 62 ++++---
tests/Makefile.am | 2 +
tests/Makefile.in | 2 +
...ap-common-setup.test => tap-driver-stderr.test} | 38 +++--
...-directive.test => tap-escape-directive-2.test} | 28 +++-
tests/tap-escape-directive.test | 4 +-
tests/tap-realtime.test | 8 +
9 files changed, 321 insertions(+), 112 deletions(-)
copy tests/{tap-common-setup.test => tap-driver-stderr.test} (56%)
copy tests/{tap-escape-directive.test => tap-escape-directive-2.test} (64%)
diff --git a/ChangeLog b/ChangeLog
index 4a72485..af6d0d4 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,104 @@
+2011-08-26 Stefano Lattarini <address@hidden>
+
+ tests: fix spurious failure for awk implementation of TAP driver
+ * tests/tap-realtime.test: The awk+shell implementation of the TAP
+ driver must "read ahead" of one line in order to catch the exit
+ status of the test script it runs. So, in the TAP-producer script
+ here, be sure to echo one "dummy" line after each TAP result line
+ in order not to cause false positives.
+
+2011-08-25 Stefano Lattarini <address@hidden>
+
+ tap/awk: allow escaping of TAP directives
+ * lib/tap-driver.sh (setup_result_obj): Handle escaping of TAP
+ directives in a way tat is (mostly) compatible by what is done
+ by the TAP::Parser module.
+ With this change, the tests `tap-escape-directive.test' and
+ `tap-escape-directive-2.test' now also pass with the shell/awk
+ implementation of the TAP driver.
+
+2011-08-25 Stefano Lattarini <address@hidden>
+
+ coverage: more about escaping of TAP directives
+ * tests/tap-escape-directive.test: Make grepping of testsuite
+ progress output slightly stricter.
+ * tests/tap-escape-directive-2.test: New test (failing with the
+ awk implementation of the TAP driver).
+ * tests/Makefile.am (tap_with_common_setup_tests): Update.
+
+2011-08-25 Stefano Lattarini <address@hidden>
+
+ tap: improve syncing between awk+shell and perl implementations
+ * lib/tap-driver.pl (stringify_test_result): Renamed ...
+ (stringify_result_obj): ... to this. Break up a clause in the
+ long "if/elsif/.../else" construct to avoid unaesthetic line
+ breaks and to be more synced with the sibling function in
+ `tap-driver.sh'. Rename the `$result', `$PASS' and `$FAIL'
+ variables to respectively `$result_obj', `$COOKED_PASS' and
+ `$COOKED_FAIL', for clarity and better syncing.
+ (handle_tap_test): Renamed ...
+ (handle_tap_result): ... to this, and change the name of the
+ `$test' local variable to `$result_obj'.
+ (extract_comment): Reimplement using the simpler `index' and
+ `substr' builtins, rather than with more advanced uses of
+ regular expressions.
+ (%test_results, @test_results): Renamed respectively ...
+ (%test_results_seen, @test_results_list): ... to these, and
+ related adjustments throughout the `TEST_RESULTS' block.
+ (main, get_global_test_result): Refactor and do some cosmetic
+ changes to make these functions clearer and better synced with
+ sibling code in `tap-driver.sh'.
+ Other minor cosmetic and typo fixes.
+ * lib/tap-driver.sh (extract_tap_comment): Remove outdated
+ "FIXME" comments.
+ (get_global_test_result): Small reordering to make it better
+ synced with its sibling function in `tap-driver.pl'.
+ (stringify_result_obj): Consistently use `result_obj' as the
+ parameter name.
+ Other minor cosmetic and typo fixes.
+
+2011-08-25 Stefano Lattarini <address@hidden>
+
+ tap/perl: don't redirect perl warnings/errors to log files
+ With this change, the test `tap-driver-stderr.test' also passes
+ with the perl implementation of the TAP driver.
+ * lib/tap-driver.pl (start): Save the original stderr into the
+ `OLDERR' file handle, and call ...
+ (trap_perl_warnings_and_errors): ... this new function, trapping
+ the `__WARN__' and `__DIE__' "pseudo-signals" to ensure that the
+ warning and error messages goes to that original stderr stream.
+ Since we are at it, be sure to prepend all possible "die" message
+ with the name of the script as given by the `$ME' variable.
+
+2011-08-25 Stefano Lattarini <address@hidden>
+
+ tap/perl: add copyright notice, version string, and emacs stuff
+ * lib/tap-driver.pl: Add proper copyright notice and bug reporting
+ address. Remove obsolete heading comments. Add trailing comments
+ for emacs w.r.t. code-formatting rules and automatic updating of
+ the date-based script version saved into the `$VERSION' variable.
+ ($VERSION): Define to the current UTC date.
+
+2011-08-24 Stefano Lattarini <address@hidden>
+
+ tap/awk: don't redirect awk stderr to log files
+ * lib/tap-driver.sh: Handle the file descriptors redirections
+ more smartly, to avoid sending error messages from awk (about
+ e.g. syntax or I/O errors) to the log files instead that to the
+ console.
+ * tests/tap-driver-stderr.test: New test, verifying the improved
+ driver behaviour. Notice that this test still fails when using
+ the perl implementation of the TAP driver.
+ * tests/Makefile.am (tap_other_tests): Update.
+
+2011-08-24 Stefano Lattarini <address@hidden>
+
+ tap/awk: remove obsolete "FIXME" comment
+ * lib/tap-driver.sh: Remove obsolete "FIXME" comment about our
+ driver losing the exit status of the tested program; this issue
+ has been solved in precedent commit `v1.11-1052-gd630a0d'. Fix
+ an unrelated typo in comments since we are at it.
+
2011-08-22 Stefano Lattarini <address@hidden>
testsuite: avoid spurious failure on Solaris
diff --git a/lib/tap-driver.pl b/lib/tap-driver.pl
index 9e1ece4..2c328d9 100755
--- a/lib/tap-driver.pl
+++ b/lib/tap-driver.pl
@@ -1,6 +1,27 @@
#! /usr/bin/env perl
-# Temporary/experimental TAP test driver for Automake.
-# TODO: should be rewritten portably (e.g., in awk or shell).
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# As a special exception to the GNU General Public License, if you
+# distribute this file as part of a program that contains a
+# configuration script generated by Autoconf, you may include it under
+# the same distribution terms that you use for the rest of that program.
+
+# This file is maintained in Automake, please report
+# bugs to <address@hidden> or send patches to
+# <address@hidden>.
# ---------------------------------- #
# Imports, static data, and setup. #
@@ -11,6 +32,8 @@ use strict;
use Getopt::Long ();
use TAP::Parser;
+my $VERSION = '2011-08-25.10'; # UTC
+
my $ME = "tap-driver.pl";
my $USAGE = <<'END';
@@ -26,8 +49,6 @@ END
my $HELP = "$ME: TAP-aware test driver for Automake testsuite harness." .
"\n" . $USAGE;
-my $VERSION = '(experimental version)';
-
# Keep this in sync with `lib/am/check.am:$(am__tty_colors)'.
my %COLOR = (
red => "\e[0;31m",
@@ -106,14 +127,15 @@ sub get_test_exit_message ();
sub get_test_results ();
sub handle_tap_bailout ($);
sub handle_tap_plan ($);
-sub handle_tap_test ($);
+sub handle_tap_result ($);
sub is_null_string ($);
sub main (@);
sub must_recheck ();
sub report ($;$);
sub start (@);
-sub stringify_test_result ($);
+sub stringify_result_obj ($);
sub testsuite_error ($);
+sub trap_perl_warnings_and_errors ();
sub write_test_results ();
sub yn ($);
@@ -134,7 +156,7 @@ sub bool_opt ($$)
}
else
{
- die "invalid argument '$val' for option '$opt'\n";
+ die "$ME: invalid argument '$val' for option '$opt'\n";
}
}
@@ -157,41 +179,42 @@ sub yn ($)
TEST_RESULTS :
{
- my (@test_results, %test_results);
+ my (@test_results_list, %test_results_seen);
sub add_test_result ($)
{
my $res = shift;
- push @test_results, $res;
- $test_results{$res} = 1;
+ push @test_results_list, $res;
+ $test_results_seen{$res} = 1;
}
sub get_test_results ()
{
- return @test_results;
+ return @test_results_list;
}
# Whether the test script should be re-run by "make recheck".
sub must_recheck ()
{
- return grep { !/^(?:XFAIL|PASS|SKIP)$/ } (keys %test_results);
+ return grep { !/^(?:XFAIL|PASS|SKIP)$/ } (keys %test_results_seen);
}
# Whether the content of the log file associated to this test should
# be copied into the "global" test-suite.log.
sub copy_in_global_log ()
{
- return grep { not $_ eq "PASS" } (keys %test_results);
+ return grep { not $_ eq "PASS" } (keys %test_results_seen);
}
# FIXME: this can certainly be improved ...
sub get_global_test_result ()
{
- my @results = keys %test_results;
- return "ERROR" if exists $test_results{"ERROR"};
- return "SKIP" if @results == 1 && $results[0] eq "SKIP";
- return "FAIL" if exists $test_results{"FAIL"};
- return "FAIL" if exists $test_results{"XPASS"};
+ return "ERROR"
+ if $test_results_seen{"ERROR"};
+ return "FAIL"
+ if $test_results_seen{"FAIL"} || $test_results_seen{"XPASS"};
+ return "SKIP"
+ if scalar keys %test_results_seen == 1 && $test_results_seen{"SKIP"};
return "PASS";
}
@@ -199,7 +222,7 @@ TEST_RESULTS :
sub write_test_results ()
{
- open RES, ">", $trs_file or die "opening $trs_file: $!\n";
+ open RES, ">", $trs_file or die "$ME: opening $trs_file: $!\n";
print RES ":global-test-result: " . get_global_test_result . "\n";
print RES ":recheck: " . yn (must_recheck) . "\n";
print RES ":copy-in-global-log: " . yn (copy_in_global_log) . "\n";
@@ -207,18 +230,32 @@ sub write_test_results ()
{
print RES ":test-result: $result\n";
}
- close RES or die "closing $trs_file: $!\n";
+ close RES or die "$ME: closing $trs_file: $!\n";
+}
+
+sub trap_perl_warnings_and_errors ()
+{
+ $SIG{__WARN__} = $SIG{__DIE__} = sub
+ {
+ # Be sure to send the warning/error message to the original stderr
+ # (presumably the console), not into the log file.
+ open STDERR, ">&", \*OLDERR;
+ die @_;
+ }
}
sub start (@)
{
# Redirect stderr and stdout to a temporary log file. Save the
# original stdout stream, since we need it to print testsuite
- # progress output.
- open LOG, ">", $log_file or die "opening $log_file: $!\n";
- open OLDOUT, ">&STDOUT" or die "duplicating stdout: $!\n";
- open STDOUT, ">&LOG" or die "redirecting stdout: $!\n";
- open STDERR, ">&LOG" or die "redirecting stderr: $!\n";
+ # progress output. Save original stderr stream, so that we can
+ # redirect warning and error messages from perl there.
+ open LOG, ">", $log_file or die "$ME: opening $log_file: $!\n";
+ open OLDOUT, ">&STDOUT" or die "$ME: duplicating stdout: $!\n";
+ open OLDERR, ">&STDERR" or die "$ME: duplicating stdout: $!\n";
+ trap_perl_warnings_and_errors;
+ open STDOUT, ">&LOG" or die "$ME: redirecting stdout: $!\n";
+ open STDERR, ">&LOG" or die "$ME: redirecting stderr: $!\n";
$parser = TAP::Parser->new ({ exec => address@hidden, merge => $cfg{merge}
});
$parser->ignore_exit(1) if $cfg{"ignore-exit"};
}
@@ -227,7 +264,7 @@ sub get_test_exit_message ()
{
my $wstatus = $parser->wait;
# Watch out for possible internal errors.
- die "couldn't get the exit ststus of the TAP producer"
+ die "$ME: couldn't get the exit ststus of the TAP producer"
unless defined $wstatus;
# Return an undefined value if the producer exited with success.
return unless $wstatus;
@@ -248,36 +285,38 @@ sub get_test_exit_message ()
}
}
-sub stringify_test_result ($)
+sub stringify_result_obj ($)
{
- my $result = shift;
- my $PASS = $cfg{"expect-failure"} ? "XPASS": "PASS";
- my $FAIL = $cfg{"expect-failure"} ? "XFAIL": "FAIL";
- if ($result->is_unplanned
- || $result->number != $testno
- || $plan_seen == LATE_PLAN)
+ my $result_obj = shift;
+ my $COOKED_PASS = $cfg{"expect-failure"} ? "XPASS": "PASS";
+ my $COOKED_FAIL = $cfg{"expect-failure"} ? "XFAIL": "FAIL";
+ if ($result_obj->is_unplanned || $result_obj->number != $testno)
{
return "ERROR";
}
- elsif (!$result->directive)
+ elsif ($plan_seen == LATE_PLAN)
{
- return $result->is_ok ? $PASS: $FAIL;
+ return "ERROR";
+ }
+ elsif (!$result_obj->directive)
+ {
+ return $result_obj->is_ok ? $COOKED_PASS: $COOKED_FAIL;
}
- elsif ($result->has_todo)
+ elsif ($result_obj->has_todo)
{
- return $result->is_actual_ok ? "XPASS" : "XFAIL";
+ return $result_obj->is_actual_ok ? "XPASS" : "XFAIL";
}
- elsif ($result->has_skip)
+ elsif ($result_obj->has_skip)
{
- return $result->is_ok ? "SKIP" : $FAIL;
+ return $result_obj->is_ok ? "SKIP" : $COOKED_FAIL;
}
- die "INTERNAL ERROR"; # NOTREACHED
+ die "$ME: INTERNAL ERROR"; # NOTREACHED
}
sub colored ($$)
{
my ($color_name, $text) = @_;
- return $COLOR{$color_name} . $text . $COLOR{'std'};
+ return $COLOR{$color_name} . $text . $COLOR{'std'};
}
sub decorate_result ($)
@@ -317,7 +356,7 @@ sub report ($;$)
}
else
{
- die "INTERNAL ERROR"; # NOTREACHED
+ die "$ME: INTERNAL ERROR"; # NOTREACHED
}
$msg .= " $explanation" if defined $explanation;
$msg .= "\n";
@@ -333,15 +372,15 @@ sub testsuite_error ($)
report "ERROR", "- $_[0]";
}
-sub handle_tap_test ($)
+sub handle_tap_result ($)
{
$testno++;
- my $test = shift;
+ my $result_obj = shift;
- my $test_result = stringify_test_result $test;
- my $string = $test->number;
+ my $test_result = stringify_result_obj $result_obj;
+ my $string = $result_obj->number;
- my $description = $test->description;
+ my $description = $result_obj->description;
$string .= " $description"
unless is_null_string $description;
@@ -349,18 +388,18 @@ sub handle_tap_test ($)
{
$string .= " # AFTER LATE PLAN";
}
- elsif ($test->is_unplanned)
+ elsif ($result_obj->is_unplanned)
{
$string .= " # UNPLANNED";
}
- elsif ($test->number != $testno)
+ elsif ($result_obj->number != $testno)
{
$string .= " # OUT-OF-ORDER (expecting $testno)";
}
- elsif (my $directive = $test->directive)
+ elsif (my $directive = $result_obj->directive)
{
$string .= " # $directive";
- my $explanation = $test->explanation;
+ my $explanation = $result_obj->explanation;
$string .= " $explanation"
unless is_null_string $explanation;
}
@@ -408,11 +447,15 @@ sub handle_tap_bailout ($)
sub extract_tap_comment ($)
{
- local $_ = shift;
- if (/^\Q$diag_string\E(.*)$/o)
+ my $line = shift;
+ if (index ($line, $diag_string) == 0)
{
- (my $comment = $1) =~ s/(?:^\s*|\s*$)//g;
- return $comment;
+ # Strip leading `$diag_string' from `$line'.
+ $line = substr ($line, length ($diag_string));
+ # And strip any leading and trailing whitespace left.
+ $line =~ s/(?:^\s*|\s*$)//g;
+ # Return what is left (if any).
+ return $line;
}
return "";
}
@@ -434,7 +477,7 @@ sub main (@)
}
elsif ($cur->is_test)
{
- handle_tap_test ($cur);
+ handle_tap_result ($cur);
}
elsif ($cur->is_bailout)
{
@@ -461,14 +504,14 @@ sub main (@)
testsuite_error (sprintf "too %s tests run (expected %d, got %d)",
$bad_amount, $planned, $run);
}
- }
- if (!$cfg{"ignore-exit"} && !$bailed_out)
- {
- my $msg = get_test_exit_message ();
- testsuite_error $msg if $msg;
+ if (!$cfg{"ignore-exit"})
+ {
+ my $msg = get_test_exit_message ();
+ testsuite_error $msg if $msg;
+ }
}
write_test_results;
- close LOG or die "closing $log_file: $!\n";
+ close LOG or die "$ME: closing $log_file: $!\n";
exit 0;
}
@@ -478,4 +521,23 @@ sub main (@)
main @ARGV;
-# vim: ft=perl ts=4 sw=4 et
+# Local Variables:
+# perl-indent-level: 2
+# perl-continued-statement-offset: 2
+# perl-continued-brace-offset: 0
+# perl-brace-offset: 0
+# perl-brace-imaginary-offset: 0
+# perl-label-offset: -2
+# cperl-indent-level: 2
+# cperl-brace-offset: 0
+# cperl-continued-brace-offset: 0
+# cperl-label-offset: -2
+# cperl-extra-newline-before-brace: t
+# cperl-merge-trailing-else: nil
+# cperl-continued-statement-offset: 2
+# eval: (add-hook 'write-file-hooks 'time-stamp)
+# time-stamp-start: "my $VERSION = "
+# time-stamp-format: "'%:y-%02m-%02d.%02H'"
+# time-stamp-time-zone: "UTC"
+# time-stamp-end: "; # UTC"
+# End:
diff --git a/lib/tap-driver.sh b/lib/tap-driver.sh
index 1255662..44317d9 100755
--- a/lib/tap-driver.sh
+++ b/lib/tap-driver.sh
@@ -23,7 +23,7 @@
# bugs to <address@hidden> or send patches to
# <address@hidden>.
-scriptversion=2011-08-21.21; # UTC
+scriptversion=2011-08-25.11; # UTC
# Make unconditional expansion of undefined variables an error. This
# helps a lot in preventing typo-related bugs.
@@ -116,13 +116,14 @@ else
fi
{
- # FIXME: this usage loses the test program exit status. We should
- # probably rewrite the awk script to use the
- # expression | getline [var]
- # idiom, which should allow us to obtain the final exit status from
- # <expression> when closing it.
- { test $merge -eq 0 || exec 2>&1; "$@"; echo $?; } \
- | LC_ALL=C ${AM_TAP_AWK-awk} \
+ { if test $merge -gt 0; then
+ exec 2>&1
+ else
+ exec 2>&3
+ fi
+ "$@"
+ echo $?
+ } | LC_ALL=C ${AM_TAP_AWK-awk} \
-v me="$me" \
-v test_script_name="$test_name" \
-v log_file="$log_file" \
@@ -133,7 +134,7 @@ fi
-v comments="$comments" \
-v diag_string="$diag_string" \
'
-# FIXME: the usages of "cat >&3" below could be optimized whne using
+# FIXME: the usages of "cat >&3" below could be optimized when using
# FIXME: GNU awk, and/on on systems that supports /dev/fd/.
# Implementation note: in what follows, `result_obj` will be an
@@ -146,7 +147,7 @@ fi
function fatal(msg)
{
- print me ": " msg | "cat >&3"
+ print me ": " msg | "cat >&2"
exit 1
}
@@ -194,35 +195,35 @@ function get_global_test_result()
{
if ("ERROR" in test_results_seen)
return "ERROR"
+ if ("FAIL" in test_results_seen || "XPASS" in test_results_seen)
+ return "FAIL"
all_skipped = 1
for (k in test_results_seen)
if (k != "SKIP")
all_skipped = 0
if (all_skipped)
return "SKIP"
- if ("FAIL" in test_results_seen || "XPASS" in test_results_seen)
- return "FAIL"
return "PASS";
}
-function stringify_result_obj(obj)
+function stringify_result_obj(result_obj)
{
- if (obj["is_unplanned"] || obj["number"] != testno)
+ if (result_obj["is_unplanned"] || result_obj["number"] != testno)
return "ERROR"
if (plan_seen == LATE_PLAN)
return "ERROR"
if (result_obj["directive"] == "TODO")
- return obj["is_ok"] ? "XPASS" : "XFAIL"
+ return result_obj["is_ok"] ? "XPASS" : "XFAIL"
if (result_obj["directive"] == "SKIP")
- return obj["is_ok"] ? "SKIP" : COOKED_FAIL;
+ return result_obj["is_ok"] ? "SKIP" : COOKED_FAIL;
if (length(result_obj["directive"]))
abort("in function stringify_result_obj()")
- return obj["is_ok"] ? COOKED_PASS : COOKED_FAIL
+ return result_obj["is_ok"] ? COOKED_PASS : COOKED_FAIL
}
function decorate_result(result)
@@ -253,10 +254,10 @@ function report(result, details)
if (length(details))
msg = msg " " details
# Output on console might be colorized.
- print decorate_result(result) msg | "cat >&3";
+ print decorate_result(result) msg
# Log the result in the log file too, to help debugging (this is
# especially true when said result is a TAP error or "Bail out!").
- print result msg;
+ print result msg | "cat >&3";
}
function testsuite_error(error_message)
@@ -293,7 +294,7 @@ function handle_tap_result()
report(stringify_result_obj(result_obj), details)
}
-# `skip_reason` should be emprty whenever planned > 0.
+# `skip_reason` should be empty whenever planned > 0.
function handle_tap_plan(planned, skip_reason)
{
planned += 0 # Avoid getting confused if, say, `planned` is "00"
@@ -328,11 +329,9 @@ function handle_tap_plan(planned, skip_reason)
function extract_tap_comment(line)
{
- # FIXME: verify there is not an off-by-one bug here.
if (index(line, diag_string) == 1)
{
# Strip leading `diag_string` from `line`.
- # FIXME: verify there is not an off-by-one bug here.
line = substr(line, length(diag_string) + 1)
# And strip any leading and trailing whitespace left.
sub("^[ \t]*", "", line)
@@ -383,7 +382,6 @@ function setup_result_obj(line)
result_obj["directive"] = ""
result_obj["explanation"] = ""
- # TODO: maybe we should allow a way to escape "#"?
if (index(line, "#") == 0)
return # No possible directive, nothing more to do.
@@ -399,6 +397,20 @@ function setup_result_obj(line)
if (!pos)
return
+ # Let`s now see if the TAP directive has been escaped. For example:
+ # escaped: ok \# SKIP
+ # not escaped: ok \\# SKIP
+ # escaped: ok \\\\\# SKIP
+ # not escaped: ok \ # SKIP
+ if (substr(line, pos, 1) == "#")
+ {
+ bslash_count = 0
+ for (i = pos; i > 1 && substr(line, i - 1, 1) == "\\"; i--)
+ bslash_count += 1
+ if (bslash_count % 2)
+ return # Directive was escaped.
+ }
+
# Strip the directive and its explanation (if any) from the test
# description.
result_obj["description"] = substr(line, 1, pos - 1)
@@ -503,7 +515,7 @@ while (1)
$0 = curline
}
# Copy any input line verbatim into the log file.
- print
+ print | "cat >&3"
# Parsing of TAP input should stop after a "Bail out!" directive.
if (bailed_out)
continue
@@ -593,7 +605,7 @@ exit 0
'
# TODO: document that we consume the file descriptor 3 :-(
-} 3>&1 >"$log_file" 2>&1
+} 3>"$log_file"
test $? -eq 0 || fatal "I/O or internal error"
diff --git a/tests/Makefile.am b/tests/Makefile.am
index c75e5d4..aee5978 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -1164,6 +1164,7 @@ tap-diagnostic.test \
tap-empty-diagnostic.test \
tap-empty.test \
tap-escape-directive.test \
+tap-escape-directive-2.test \
tap-exit.test \
tap-signal.test \
tap-fancy.test \
@@ -1234,6 +1235,7 @@ tap-common-setup.test \
tap-bad-prog.tap \
tap-basic.test \
tap-diagnostic-custom.test \
+tap-driver-stderr.test \
tap-doc.test \
tap-doc2.test \
tap-more.test \
diff --git a/tests/Makefile.in b/tests/Makefile.in
index 1f2565a..8cf3bc4 100644
--- a/tests/Makefile.in
+++ b/tests/Makefile.in
@@ -1391,6 +1391,7 @@ tap-diagnostic.test \
tap-empty-diagnostic.test \
tap-empty.test \
tap-escape-directive.test \
+tap-escape-directive-2.test \
tap-exit.test \
tap-signal.test \
tap-fancy.test \
@@ -1458,6 +1459,7 @@ tap-common-setup.test \
tap-bad-prog.tap \
tap-basic.test \
tap-diagnostic-custom.test \
+tap-driver-stderr.test \
tap-doc.test \
tap-doc2.test \
tap-more.test \
diff --git a/tests/tap-common-setup.test b/tests/tap-driver-stderr.test
similarity index 56%
copy from tests/tap-common-setup.test
copy to tests/tap-driver-stderr.test
index 92ae3c4..396624e 100755
--- a/tests/tap-common-setup.test
+++ b/tests/tap-driver-stderr.test
@@ -14,30 +14,36 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Auxiliary test to set up common data used by many tests on TAP support.
+# Unit test on TAP driver:
+# - error messages from awk/shell/perl goes to the console
+required=non-root
parallel_tests=yes
. ./defs || Exit 1
-cat >> configure.in << END
-AC_OUTPUT
-END
+fetch_tap_driver
-cat > Makefile.am << 'END'
-TEST_LOG_DRIVER = $(srcdir)/tap-driver
-TEST_LOG_COMPILER = cat
-TESTS = all.test
-END
+tst=zardoz
-$ACLOCAL
-$AUTOCONF
-$AUTOMAKE
+for suf in trs log; do
-./configure
+ rm -f $tst.log $tst.trs
+ touch $tst.$suf
+ chmod a-w $tst.$suf
-rm -rf autom4te*.cache
+ st=0
+ ./tap-driver --test-name $tst --log-file $tst.log --trs-file $tst.trs \
+ -- sh -c 'echo 1..1; echo ok 1; echo "Hello, World!"' \
+ >stdout 2>stderr && st=1
+ cat stdout
+ cat stderr >&2
+ cat $tst.log || :
+ cat $tst.trs || :
+ test $st -eq 0
-# So that the data files we've created won't be removed at exit.
-keep_testdirs=yes
+ $FGREP 'Hello, World!' stderr stdout && Exit 1
+ $FGREP $tst.$suf stderr
+
+done
:
diff --git a/tests/tap-escape-directive.test b/tests/tap-escape-directive-2.test
similarity index 64%
copy from tests/tap-escape-directive.test
copy to tests/tap-escape-directive-2.test
index dc03197..e0dd37b 100755
--- a/tests/tap-escape-directive.test
+++ b/tests/tap-escape-directive-2.test
@@ -23,17 +23,33 @@ parallel_tests=yes
. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
cat > all.test <<'END'
-1..2
-ok \# TODO
-ok \# SKIP
+1..8
+
+not ok \ # TODO
+ok \ # SKIP
+
+not ok \\# TODO
+ok \\# SKIP
+
+ok \\\# TODO
+ok \\\# SKIP
+
+not ok \\\\\\\\\\# TODO
+ok \\\\\\\\\\# SKIP
END
$MAKE check >stdout || { cat stdout; Exit 1; }
cat stdout
-count_test_results total=2 pass=2 fail=0 xpass=0 xfail=0 skip=0 error=0
+count_test_results total=8 pass=2 fail=0 xpass=0 xfail=3 skip=3 error=0
-grep '^PASS: all\.test 1 .*TODO' stdout
-grep '^PASS: all\.test 2 .*SKIP' stdout
+grep '^XFAIL: all\.test 1 .*# TODO' stdout
+grep '^SKIP: all\.test 2 .*# SKIP' stdout
+grep '^XFAIL: all\.test 3 .*# TODO' stdout
+grep '^SKIP: all\.test 4 .*# SKIP' stdout
+grep '^PASS: all\.test 5 .*# TODO' stdout
+grep '^PASS: all\.test 6 .*# SKIP' stdout
+grep '^XFAIL: all\.test 7 .*# TODO' stdout
+grep '^SKIP: all\.test 8 .*# SKIP' stdout
:
diff --git a/tests/tap-escape-directive.test b/tests/tap-escape-directive.test
index dc03197..194db6a 100755
--- a/tests/tap-escape-directive.test
+++ b/tests/tap-escape-directive.test
@@ -33,7 +33,7 @@ cat stdout
count_test_results total=2 pass=2 fail=0 xpass=0 xfail=0 skip=0 error=0
-grep '^PASS: all\.test 1 .*TODO' stdout
-grep '^PASS: all\.test 2 .*SKIP' stdout
+grep '^PASS: all\.test 1 .*# TODO' stdout
+grep '^PASS: all\.test 2 .*# SKIP' stdout
:
diff --git a/tests/tap-realtime.test b/tests/tap-realtime.test
index 4a38f14..61476c3 100755
--- a/tests/tap-realtime.test
+++ b/tests/tap-realtime.test
@@ -47,15 +47,23 @@ echo 1..3
# FIXME: creative quoting to placate maintainer-check
sleep="sleep "3
+# The awk+shell implementation of the TAP driver must "read ahead" of one
+# line in order to catch the exit status of the test script it runs. So
+# be sure to echo one "dummy" line after each result line in order not to
+# cause false positives.
+
echo ok 1 - foo
+echo DUMMY
$sleep
test -f ok-1 || { echo 'Bail out!'; exit 1; }
echo ok 2 - bar
+echo DUMMY
$sleep
test -f ok-2 || { echo 'Bail out!'; exit 1; }
echo ok 3 - baz
+echo DUMMY
$sleep
test -f ok-3 || { echo 'Bail out!'; exit 1; }
hooks/post-receive
--
GNU Automake
| [Prev in Thread] |
Current Thread |
[Next in Thread] |
- [Automake-commit] [SCM] GNU Automake branch, testsuite-work, updated. v1.11-1247-g449a81c,
Stefano Lattarini <=