[Automake-commit] [SCM] GNU Automake branch, test-protocols, updated. v1.11-1041-g28964e4

[Stefano Lattarini]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "GNU Automake".

http://git.sv.gnu.org/gitweb/?p=automake.git;a=commitdiff;h=28964e4787b17f0211f9b2aab2776cee163fdb30

The branch, test-protocols has been updated
       via  28964e4787b17f0211f9b2aab2776cee163fdb30 (commit)
       via  135cfa3b653af3c4cc9bec95f813237263ae02ef (commit)
       via  38f9c6c7a64a411d2655f985b9929a3bbb2d3beb (commit)
       via  5c74dedc0e0ed50680180259133a02086d5695e6 (commit)
      from  11d8ed522235c2e5c705d6345e7cd55f2a47b316 (commit)

Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.

- Log -----------------------------------------------------------------
commit 28964e4787b17f0211f9b2aab2776cee163fdb30
Author: Stefano Lattarini <address@hidden>
Date:   Sun Aug 21 12:25:39 2011 +0200

    coverage: ambiguous use of TAP "TODO"/"SKIP" directives
    
    * tests/tap-ambiguous-directive.test: New test.
    * tests/tap-todo-skip-together.test: Reference it in heading
    comments.
    * tests/Makefile.am (tap_with_common_setup_tests): Update.

commit 135cfa3b653af3c4cc9bec95f813237263ae02ef
Author: Stefano Lattarini <address@hidden>
Date:   Sun Aug 21 11:01:51 2011 +0200

    coverage: non-directive comments in TAP results are preserved
    
    * tests/tap-result-comment.test: New test.
    * tests/Makefile.am (tap_with_common_setup_tests): Update.

commit 38f9c6c7a64a411d2655f985b9929a3bbb2d3beb
Author: Stefano Lattarini <address@hidden>
Date:   Sat Aug 20 18:30:12 2011 +0200

    coverage: TAP result number followed by non-whitespace word boundary
    
    * tests/tap-number-wordboundary.test: New test.
    * tests/Makefile.am (tap_with_common_setup_tests): Update.

commit 5c74dedc0e0ed50680180259133a02086d5695e6
Author: Stefano Lattarini <address@hidden>
Date:   Sat Aug 20 17:32:29 2011 +0200

    coverage: a TAP result numbered "0" is always out-of-order
    
    * tests/tap-test-number-0.test: New, checks that a test result
    numbered as 0 is to be considered out-of-order; this is
    consistent with the behaviour of the `prove' utility.
    * tests/Makefile.am (tap_with_common_setup_tests): Update.

-----------------------------------------------------------------------

Summary of changes:
 ChangeLog                                          |   28 +++++
 tests/Makefile.am                                  |    4 +
 tests/Makefile.in                                  |    4 +
 ...e-numbers.test => tap-ambiguous-directive.test} |   38 +++----
 tests/tap-number-wordboundary.test                 |  111 ++++++++++++++++++++
 ...gative-numbers.test => tap-result-comment.test} |   40 +++----
 ...an-leading-zero.test => tap-test-number-0.test} |   46 +++++----
 tests/tap-todo-skip-together.test                  |    1 +
 8 files changed, 210 insertions(+), 62 deletions(-)
 copy tests/{tap-negative-numbers.test => tap-ambiguous-directive.test} (66%)
 create mode 100755 tests/tap-number-wordboundary.test
 copy tests/{tap-negative-numbers.test => tap-result-comment.test} (63%)
 copy tests/{tap-plan-leading-zero.test => tap-test-number-0.test} (56%)

diff --git a/ChangeLog b/ChangeLog
index c64cf6e..810cee0 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,31 @@
+2011-08-21  Stefano Lattarini  <address@hidden>
+
+       coverage: ambiguous use of TAP "TODO"/"SKIP" directives
+       * tests/tap-ambiguous-directive.test: New test.
+       * tests/tap-todo-skip-together.test: Reference it in heading
+       comments.
+       * tests/Makefile.am (tap_with_common_setup_tests): Update.
+
+2011-08-21  Stefano Lattarini  <address@hidden>
+
+       coverage: non-directive comments in TAP results are preserved
+       * tests/tap-result-comment.test: New test.
+       * tests/Makefile.am (tap_with_common_setup_tests): Update.
+
+2011-08-20  Stefano Lattarini  <address@hidden>
+
+       coverage: TAP result number followed by non-whitespace word boundary
+       * tests/tap-number-wordboundary.test: New test.
+       * tests/Makefile.am (tap_with_common_setup_tests): Update.
+
+2011-08-20  Stefano Lattarini  <address@hidden>
+
+       coverage: a TAP result numbered "0" is always out-of-order
+       * tests/tap-test-number-0.test: New, checks that a test result
+       numbered as 0 is to be considered out-of-order; this is
+       consistent with the behaviour of the `prove' utility.
+       * tests/Makefile.am (tap_with_common_setup_tests): Update.
+
 2011-08-20  Stefano Lattarini  <address@hidden>
 
        cosmetics: fix botched indentation in perl TAP driver
diff --git a/tests/Makefile.am b/tests/Makefile.am
index e7f8570..cd6ed2c 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -1142,6 +1142,7 @@ testsuite-summary-count-many.log: 
extract-testsuite-summary
 # `tap-common-setup.test', and sources the `tap-setup.sh' helper
 # script.
 tap_with_common_setup_tests = \
+tap-ambiguous-directive.test \
 tap-autonumber.test \
 tap-bailout.test \
 tap-bailout-and-logging.test \
@@ -1174,6 +1175,7 @@ tap-no-spurious-summary.test \
 tap-no-spurious-numbers.test \
 tap-no-spurious.test \
 tap-not-ok-skip.test \
+tap-number-wordboundary.test \
 tap-numeric-description.test \
 tap-negative-numbers.test \
 tap-numbers-leading-zero.test \
@@ -1198,7 +1200,9 @@ tap-planskip-badexit.test \
 tap-planskip-bailout.test \
 tap-planskip-later-errors.test \
 tap-realtime.test \
+tap-test-number-0.test \
 tap-recheck-logs.test \
+tap-result-comment.test \
 tap-todo-skip-together.test \
 tap-todo-skip-whitespace.test \
 tap-todo-skip.test \
diff --git a/tests/Makefile.in b/tests/Makefile.in
index c6ca939..ed955ac 100644
--- a/tests/Makefile.in
+++ b/tests/Makefile.in
@@ -1383,6 +1383,7 @@ $(parallel_tests)
 # `tap-common-setup.test', and sources the `tap-setup.sh' helper
 # script.
 tap_with_common_setup_tests = \
+tap-ambiguous-directive.test \
 tap-autonumber.test \
 tap-bailout.test \
 tap-bailout-and-logging.test \
@@ -1415,6 +1416,7 @@ tap-no-spurious-summary.test \
 tap-no-spurious-numbers.test \
 tap-no-spurious.test \
 tap-not-ok-skip.test \
+tap-number-wordboundary.test \
 tap-numeric-description.test \
 tap-negative-numbers.test \
 tap-numbers-leading-zero.test \
@@ -1439,7 +1441,9 @@ tap-planskip-badexit.test \
 tap-planskip-bailout.test \
 tap-planskip-later-errors.test \
 tap-realtime.test \
+tap-test-number-0.test \
 tap-recheck-logs.test \
+tap-result-comment.test \
 tap-todo-skip-together.test \
 tap-todo-skip-whitespace.test \
 tap-todo-skip.test \
diff --git a/tests/tap-negative-numbers.test 
b/tests/tap-ambiguous-directive.test
similarity index 66%
copy from tests/tap-negative-numbers.test
copy to tests/tap-ambiguous-directive.test
index 9fdee87..78f9684 100755
--- a/tests/tap-negative-numbers.test
+++ b/tests/tap-ambiguous-directive.test
@@ -15,38 +15,36 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 # TAP support:
-#  - don't spuriously recognize negative TAP result numbers, but correctly
-#    interpret them as test descriptions instead
+#  - handling of "ambiguous" TODO and SKIP directives
+# See also related test 'tap-todo-skip-together.test'.
 
 parallel_tests=yes
 . ./defs || Exit 1
 
 . "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
 
-cat > all.test <<'END'
-1..7
-ok -1
-not ok -3
-ok -2 # SKIP
-not ok -5 # TODO
-ok -04 # TODO
-ok -121
-not ok -50000
+cat > all.test <<END
+1..6
+ok 1 # foo SKIP
+not ok 2 # bar TODO
+ok 3 # :SKIP
+not ok 4 # :TODO
+ok 5 # SKIP SKIP
+not ok 6 # TODO TODO
 END
 
 $MAKE check >stdout && { cat stdout; Exit 1; }
 cat stdout
 
-count_test_results total=7 pass=2 fail=2 xpass=1 xfail=1 skip=1 error=0
+count_test_results total=6 pass=2 fail=2 xpass=0 xfail=1 skip=1 error=0
 
-cat > exp <<'END'
-PASS: all.test 1 -1
-FAIL: all.test 2 -3
-SKIP: all.test 3 -2 # SKIP
-XFAIL: all.test 4 -5 # TODO
-XPASS: all.test 5 -04 # TODO
-PASS: all.test 6 -121
-FAIL: all.test 7 -50000
+cat > exp << 'END'
+PASS: all.test 1 # foo SKIP
+FAIL: all.test 2 # bar TODO
+PASS: all.test 3 # :SKIP
+FAIL: all.test 4 # :TODO
+SKIP: all.test 5 # SKIP SKIP
+XFAIL: all.test 6 # TODO TODO
 END
 
 $FGREP ': all.test' stdout > got
diff --git a/tests/tap-number-wordboundary.test 
b/tests/tap-number-wordboundary.test
new file mode 100755
index 0000000..5235738
--- /dev/null
+++ b/tests/tap-number-wordboundary.test
@@ -0,0 +1,111 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# TAP support:
+#  - TAP result numbers terminated by a non-whitespace "word boundary"
+#    character are recognized
+
+parallel_tests=yes
+. ./defs || Exit 1
+
+. "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
+
+cat > all.test <<'END'
+1..5
+ok 1: foo1
+not ok 2- foo2
+ok 3@ foo3 # SKIP
+not ok 4?&%$ foo4 # TODO
+ok 5"`!! foo5 # TODO
+END
+
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=5 pass=1 fail=1 skip=1 xpass=1 xfail=1 error=0
+
+$EGREP '^PASS: all\.test 1 ?: foo1$' stdout
+$EGREP '^FAIL: all\.test 2 ?- foo2$' stdout
+$EGREP '^SKIP: all\.test 3 ?@ foo3 # SKIP$' stdout
+$EGREP '^XFAIL: all\.test 4 ?\?&%\$ foo4 # TODO$' stdout
+$EGREP '^XPASS: all\.test 5 ?"`!! foo5 # TODO$' stdout
+
+cat > all.test <<'END'
+1..3
+ok 2: quux
+not ok 1!
+ok 476$ # SKIP
+END
+
+$MAKE check >stdout && { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=3 pass=0 fail=0 skip=0 xpass=0 xfail=0 error=3
+
+$EGREP '^ERROR: all\.test 2 ?: quux # OUT-OF-ORDER \(expecting 1\)$' stdout
+$EGREP '^ERROR: all\.test 1 ?! # OUT-OF-ORDER \(expecting 2\)$' stdout
+$EGREP '^ERROR: all\.test 476 ?\$ # OUT-OF-ORDER \(expecting 3\)$' stdout
+
+cat > punctuation <<'END'
+'
+"
+`
+#
+$
+!
+\
+/
+&
+%
+(
+)
+|
+^
+~
+?
+*
++
+-
+,
+:
+;
+=
+<
+>
+@
+[
+]
+{
+}
+\
+END
+
+planned=`wc -l <punctuation`
+
+echo 1..$planned > all.test
+awk '{print "ok " NR $0 }' punctuation >> all.test
+# For debugging.
+cat all.test
+# Sanity check.
+grep "^ok 1'$" all.test || framework_failure_ "creating all.test"
+
+$MAKE check >stdout || { cat stdout; Exit 1; }
+cat stdout
+
+count_test_results total=$planned pass=$planned \
+                   fail=0 skip=0 xpass=0 xfail=0 error=0
+
+:
diff --git a/tests/tap-negative-numbers.test b/tests/tap-result-comment.test
similarity index 63%
copy from tests/tap-negative-numbers.test
copy to tests/tap-result-comment.test
index 9fdee87..0cfee01 100755
--- a/tests/tap-negative-numbers.test
+++ b/tests/tap-result-comment.test
@@ -15,41 +15,37 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 # TAP support:
-#  - don't spuriously recognize negative TAP result numbers, but correctly
-#    interpret them as test descriptions instead
+#  - non-directive comments in TAP results are kept verbatim
 
 parallel_tests=yes
 . ./defs || Exit 1
 
 . "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
 
-cat > all.test <<'END'
-1..7
-ok -1
-not ok -3
-ok -2 # SKIP
-not ok -5 # TODO
-ok -04 # TODO
-ok -121
-not ok -50000
+cat > all.test <<END
+1..5
+ok 1 # TDO
+not ok 2 # SKP
+ok #SIKP${tab}${sp}
+not ok #    TOD${tab}
+ok 5 # ${tab}${tab}TOOD${tab}${sp}${sp}
 END
 
 $MAKE check >stdout && { cat stdout; Exit 1; }
 cat stdout
 
-count_test_results total=7 pass=2 fail=2 xpass=1 xfail=1 skip=1 error=0
+count_test_results total=5 pass=3 fail=2 xpass=0 xfail=0 skip=0 error=0
 
-cat > exp <<'END'
-PASS: all.test 1 -1
-FAIL: all.test 2 -3
-SKIP: all.test 3 -2 # SKIP
-XFAIL: all.test 4 -5 # TODO
-XPASS: all.test 5 -04 # TODO
-PASS: all.test 6 -121
-FAIL: all.test 7 -50000
-END
+# Don't be too strict about trailing white space.
+$FGREP ': all.test' stdout | sed "s/[$sp$tab]*$//" > got
 
-$FGREP ': all.test' stdout > got
+cat > exp <<END
+PASS: all.test 1 # TDO
+FAIL: all.test 2 # SKP
+PASS: all.test 3 #SIKP
+FAIL: all.test 4 #    TOD
+PASS: all.test 5 # ${tab}${tab}TOOD
+END
 
 cat exp
 cat got
diff --git a/tests/tap-plan-leading-zero.test b/tests/tap-test-number-0.test
similarity index 56%
copy from tests/tap-plan-leading-zero.test
copy to tests/tap-test-number-0.test
index 4c54649..abce60e 100755
--- a/tests/tap-plan-leading-zero.test
+++ b/tests/tap-test-number-0.test
@@ -15,8 +15,7 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 # TAP support:
-#  - TAP plans with numbers having leading zeroes, as in "1..01"
-#  - TAP "SKIP" plans with multiple zeroes, as in "1..00 # SKIP"
+#  - a test result numbered as 0 is to be considered out-of-order
 # This is consistent with the behaviour of the `prove' utility.
 
 parallel_tests=yes
@@ -25,33 +24,40 @@ parallel_tests=yes
 . "$testsrcdir"/tap-setup.sh || fatal_ "sourcing tap-setup.sh"
 
 cat > a.test <<END
-1..01
-ok 1
+1..1
+ok 0
 END
 
 cat > b.test <<END
-1..0002
-ok 1
-ok 2
+1..1
+not ok 0
 END
 
-echo 1..010 > c.test
-for i in 1 2 3 4 5 6 7 8 9 10; do echo ok $i; done >> c.test
+cat > c.test <<END
+1..1
+ok 0 foo # SKIP
+END
 
-echo 1..00100 > d.test
-for i in 1 2 3 4 5 6 7 8 9 10; do
-  for j in 1 2 3 4 5 6 7 8 9 10; do
-    echo ok
-  done
-done >> d.test
+cat > d.test <<END
+1..1
+not ok 0 bar # TODO
+END
 
-echo 1..00 > e.test
-echo '1..000 # SKIP' > f.test
+cat > e.test <<END
+1..1
+ok 0 # TODO
+END
 
-env TESTS='a.test b.test c.test d.test e.test f.test' \
-  $MAKE -e check >stdout || { cat stdout; Exit 1; }
+TESTS='a.test b.test c.test d.test e.test' $MAKE -e check >stdout \
+  && { cat stdout; Exit 1; }
 cat stdout
 
-count_test_results total=115 pass=113 xfail=0 skip=2 fail=0 xpass=0 error=0
+count_test_results total=5 pass=0 fail=0 xpass=0 xfail=0 skip=0 error=5
+
+grep '^ERROR: a\.test 0 # OUT-OF-ORDER (expecting 1)$' stdout
+grep '^ERROR: b\.test 0 # OUT-OF-ORDER (expecting 1)$' stdout
+grep '^ERROR: c\.test 0 foo # OUT-OF-ORDER (expecting 1)$' stdout
+grep '^ERROR: d\.test 0 bar # OUT-OF-ORDER (expecting 1)$' stdout
+grep '^ERROR: e\.test 0 # OUT-OF-ORDER (expecting 1)$' stdout
 
 :
diff --git a/tests/tap-todo-skip-together.test 
b/tests/tap-todo-skip-together.test
index 14b8ed5..0f4a943 100755
--- a/tests/tap-todo-skip-together.test
+++ b/tests/tap-todo-skip-together.test
@@ -16,6 +16,7 @@
 
 # TAP support:
 #  - TODO and SKIP directives on the same line: the first one wins
+# See also related test 'tap-ambiguous-directive.test'.
 
 parallel_tests=yes
 . ./defs || Exit 1


hooks/post-receive
-- 
GNU Automake