[Automake-commit] [SCM] GNU Automake branch, GSoC/experimental/test-results-work, created. v1.11-914-gd0ac0b5

[Stefano Lattarini]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "GNU Automake".

http://git.sv.gnu.org/gitweb/?p=automake.git;a=commitdiff;h=d0ac0b5c7fa5a957f457532768245503e219a4ba

The branch, GSoC/experimental/test-results-work has been created
        at  d0ac0b5c7fa5a957f457532768245503e219a4ba (commit)

- Log -----------------------------------------------------------------
commit d0ac0b5c7fa5a957f457532768245503e219a4ba
Author: Stefano Lattarini <address@hidden>
Date:   Mon Aug 1 19:10:41 2011 +0200

    testsuite: separate the only failing check of an xfailing test
    
    * tests/plan-bad-prog.test: Move the only failing check of this
    test (i.e., the one about the total number of "ERROR" outcomes)
    into ...
    * tests/plan-bad-prog2.test: ... this new test.
    * tests/Makefile.am (XFAIL_TESTS): Remove `plan-bad-prog.test',
    add `plan-bad-prog2.test'.
    (tap_other_tests): Add `plan-bad-prog2.test'.
    (plan-bad-prog2.log): Depend on `plan-bad-prog.test'.

commit 236421ec86d541ba2a3a1446204b2dade81816ed
Author: Stefano Lattarini <address@hidden>
Date:   Mon Aug 1 18:43:30 2011 +0200

    testsuite: fix spurious errors in an xfailing test on TAP support
    
    * tests/plan-bad-prog.test: Fix typo in the name of the test
    being made unreadable.  Correct the wording of the potential
    skip message, and simplify the condition under which the test
    is to be skipped.  Escape literal dots in grep regexps.

commit c2abfde35c37e8c210cba412fe9b926241fe6ef7
Author: Stefano Lattarini <address@hidden>
Date:   Mon Aug 1 18:32:17 2011 +0200

    testsuite: better granularity in a couple of tests on TAP support
    
    * tests/tap-summary-aux.sh: New auxiliary script, filled with code
    moved out from ...
    * tests/tap-summary.test: ... this test, from which the checks on
    colored testsuite have further been moved into ...
    * tests/tap-summary-color.test: ... this new test, which in turn
    * tests/Makefile.am (tap_other_tests): Add `tap-summary-color.test'.
    (EXTRA_DIST): Distribute `tap-summary-aux.sh'
    (tap-summary.log): Depend on `tap-summary-aux.sh'.
    (tap-summary-color.log): Likewise.

commit 86009695fdbb9d92828d3e85c9511b93f8471381
Author: Stefano Lattarini <address@hidden>
Date:   Mon Aug 1 16:14:44 2011 +0200

    testsuite: optimize tests on TAP for speed and against duplication
    
    The creation and configuration of common files and data used by
    many tests on TAP is, with this change, factored out into a new
    dedicated auxiliary test, from which those tests will depend upon.
    This reduces code duplication in tests and, more importantly,
    offers a noticeable speedup in the involved tests (30-40%).
    
    * tests/tap-common-setup.test: New test, setting up the common
    files and data used by various tests on TAP support.
    * tests/tap-setup.sh: New file, to be sourced by tests wanting to
    bring in data generated by `tap-common-setup.test'.
    * tests/tap-autonumber.test: Update to use the pre-computed data
    files.
    * tests/tap-bailout.test: Likewise.
    * tests/tap-color.test: Likewise.
    * tests/tap-deps.test: Likewise.
    * tests/tap-diagnostic.test: Likewise.
    * tests/tap-empty-diagnostic.test: Likewise.
    * tests/tap-empty.test: Likewise.
    * tests/tap-escape-directive.test: Likewise.
    * tests/tap-exit.test: Likewise.
    * tests/tap-signal.test: Likewise.
    * tests/tap-fancy.test: Likewise.
    * tests/tap-fancy2.test: Likewise.
    * tests/tap-global-log.test: Likewise.
    * tests/tap-global-result.test: Likewise.
    * tests/tap-html.test: Likewise.
    * tests/tap-log.test: Likewise.
    * tests/tap-merge-stdout-stderr.test: Likewise.
    * tests/tap-no-merge-stdout-stderr.test: Likewise.
    * tests/tap-message-0.test: Likewise.
    * tests/tap-no-disable-hard-error.test: Likewise.
    * tests/tap-no-spurious-summary.test: Likewise.
    * tests/tap-no-spurious.test: Likewise.
    * tests/tap-not-ok-skip.test: Likewise.
    * tests/tap-numeric-description.test: Likewise.
    * tests/tap-out-of-order.test: Likewise.
    * tests/tap-passthrough.test: Likewise.
    * tests/tap-passthrough-exit.test: Likewise.
    * tests/tap-plan.test: Likewise.
    * tests/tap-plan-corner.test: Likewise.
    * tests/tap-plan-corner2.test: Likewise.
    * tests/tap-plan-errors.test: Likewise.
    * tests/tap-realtime.test: Likewise.
    * tests/tap-recheck-logs.test: Likewise.
    * tests/tap-skip-whole.test: Likewise.
    * tests/tap-todo-skip-together.test: Likewise.
    * tests/tap-todo-skip-whitespace.test: Likewise.
    * tests/tap-todo-skip.test: Likewise.
    * tests/tap-unplanned.test: Likewise.
    * tests/tap-with-and-without-number.test: Likewise.
    * tests/tap-xfail-tests.test: Likewise.
    * tests/tap-skip-whole-whitespace.test: Likewise, and remove
    redundant definitions of `$sp' and `$tab' (they are already
    defined in `tests/defs').
    * tests/tap-whitespace-normalization.test: Likewise.
    * tests/Makefile.am (TESTS): Update.
    (EXTRA_DIST): Distribute `tap-setup.sh'.
    (tap_with_common_setup_tests, tap_with_common_setup_logs)): New
    variables, holding respectively the list of tests using the files
    pre-computed by `tap-common-setup.test', and the list of their
    corresponding log files.
    (tap_other_tests): New variable, holding the list of other tests
    on TAP support.
    ($(tap_with_common_setup_logs)): Depend on `tap-common-setup.log'
    and `tap-setup.sh'.

commit bd16d593264f77ab6b048589a557e5eadbcc8bb4
Author: Stefano Lattarini <address@hidden>
Date:   Thu Jul 28 12:25:22 2011 +0200

    test harness: use new `.trs' files to hold test metadata
    
    With this change, the test harness will keep test metadata in
    dedicated `.trs' files, instead of having them embedded into the
    `.log' files.  This allows for easier forward-compatibility and
    extension of test metadata, and for more flexibility in the
    format of the `.log' files.  Note that this change makes the
    `:end-metadata:' field obsolete.
    
    * doc/automake.texi (Parallel Test Harness, Log files generation
    and test results recording): Document the new APIs and behaviour;
    some related minor rewordings and fixlets.
    * automake.in (handle_tests): When bringing in the content of
    `check2.am', substitute %BASE% with the basename of the `.log'
    file being created by a rule.  Add the generated `.trs' files
    to the list of files to be cleaned by "make mostlyclean".
    * lib/am/check.am (am__test_driver_flags): Rename ...
    (am__common_driver_flags): ... to this, and remove the flags
    `--test-name' and `--log-file' from it: they are now define in
    the proper rules in `check2.am'.
    (am__TEST_BASES): New internal variable, holding the names of
    the tests, with any registered extension removed.
    (am__stealth_MAKE): New internal variable, can be used instead of
    $(MAKE) in recipes requiring a recursive call to make, but which
    are not intended to be executed by "make -n".
    (.log.trs): New suffix rule, to recover from deletion of `.trs'
    files.
    ($(TEST_SUITE_LOG)): Almost completely rewritten to follow the
    new API of "test logs in `.log' files, test metadata in `.trs'
    files".  It goes to some length to work correctly in face of
    unreadable or missing `.log' and `.trs' files, and to error out
    with proper error messages when this is not possible.
    [%?PARALLEL_TESTS%] (check-TESTS): Also remove relevant "stale"
    `.trs' files (in addition to `.log files) before remaking the
    $(TEST_SUITE_LOG).
    (recheck, recheck-html): Look for the `:recheck:' field in the
    `.trs' files, not in the `.log' files.
    * lib/am/check2.am (?GENERIC?%EXT%.log, ?!GENERIC?%OBJ%): Adjust
    the call to the test driver, in particularly passing the new
    option `--trs-file'.
    [%am__EXEEXT%] (?GENERIC?%EXT%$(EXEEXT).log): Likewise.
    * lib/tap-driver ($USAGE): Adjust the help screen.
    (Getopt::Long::GetOptions): Handle the `--trs-file' option,
    through the use of ...
    ($trs_file): ... this new global variable.
    (finish): Write metadata for the test run to `$trs_file' rather
    then to `$log_file', through the use of ...
    (write_test_results): ... this new function.
    * lib/test-driver (print_usage): Update the help screen.
    (Option parsing): Handle the `--trs-file' option, through the
    use of ...
    ($resfile): ... this new global variable.
    (Main code): Write metadata for the test run to `$trsfile' rather
    than to `$logfile'.
    Minor related adjustments to comments.
    * tests/.gitignore: Ignore `*.trs' files.
    * tests/parallel-tests-unreadable-log.test: Moved ...
    * tests/parallel-tests-unreadable.test: ... to this, and extended
    to also check the semantics for unreadable `.trs' files.
    * tests/test-driver-end-metadata.test: Deleted as obsolete.
    * tests/test-driver-metadata-no-leading-space.test: Likewise.
    * tests/test-driver-global-log.test: Renamed ...
    * tests/test-metadata-global-log.test: ... to this, and modified
    as to verify the new APIs and semantics.
    * tests/test-driver-recheck: Renamed ...
    * tests/test-metadata-recheck.test: ... to this, and modified
    likewise.
    * tests/parallel-tests-once.test: New test.
    * tests/parallel-tests-make-n.test: Likewise.
    * test-metadata-results.test: Likewise.
    * test-missing.test: Likewise.
    * test-missing2.test: Likewise.
    * test-trs-basic.test: Likewise.
    * test-trs-recover.test: Likewise.
    * test-trs-recover2.test: Likewise.
    * tests/Makefile.am (TESTS): Update.

commit 2534e35dbe9044fb8386159be1838cef5330bfa2
Author: Stefano Lattarini <address@hidden>
Date:   Tue Jul 26 14:51:20 2011 +0200

    test harness: allow more metadata in log files
    
    This change reworks and improves the parallel test harness to use
    more specialized reStructuredText fields in the log files (instead
    of relying on specially-placed of "magic lines" and more indirect
    semantical formatting); the new fields are the following:
     - ":recheck:": tell whether the associated test will have to be
       re-run by "make recheck";
     - ":copy-in-global-log:": tell whether the content of the log
       file should be copied in the "global log" `test-suite.log';
     - ":end-metadata:", which inhibits the scanning of the rest of
       the log file (for what concerns test metadata).
    Also, the special `:test-result:' value "END" has been removed,
    superseded by the new `:end-metadata:' field.
    
    * doc/automake.texi (Log files generation and test results
    recording): Document the new API and semantics.  Remove or fix
    some obsolete comments.
    * lib/am/check.am ($(TEST_SUITE_LOG), recheck, recheck-html):
    Adjust comments and code.
    * lib/tap-driver (finish): Adjust, with the help of ...
    (must_recheck, copy_in_global_log): ... these new functions.
    * lib/test-driver (Main code): Adjust, with the help of ...
    ($recheck, $gcopy): ... these new variables.
    * tests/trivial-test-driver: Update to obey the new APIs.
    * tests/test-driver-recheck.test: Likewise.
    * tests/test-driver-global-log.test: Likewise.
    * tests/tap-passthrough.test: Relax the test, avoiding to check
    what is written in `test-suite.log'; such check has been moved ...
    * tests/tap-global-log.test: ... in this new test, and extended.
    * tests/test-driver-metadata-no-leading-space.test: New test.
    * tests/test-driver-end-test-results.test: Removed, it checked
    the old APIs; superseded by ...
    * tests/test-driver-end-metadata.test: ... this new test.
    * tests/tap-log.test: Improve syncing with ...
    * tests/test-log.test: ... this new test.
    * tests/parallel-tests.test: Remove some duplication w.r.t. this
    last new test.  Updated heading comments.
    * tests/Makefile.am (TESTS): Update.

commit 46eb97e4ba388249efcf21b1d3fa129adb2783b3
Author: Stefano Lattarini <address@hidden>
Date:   Wed Jul 27 16:34:30 2011 +0200

    tests: fix a spurious failure with non-bash shells
    
    * tests/tap-fancy2.test: Remove an unportable use of backslashes
    with the `echo' builtin, which was causing some shells (among them
    bash and Debian /bin/ksh), to print a `\\' string, while other
    shells (among them zsh, dash, and Solaris /bin/sh and /bin/ksh)
    were unexpectedly printing a single `\' character.  Since we are
    at it, add a sanity check to ensure that this issue does not
    resurface.

commit 3e4f4a9390e3e0cc5abeeb53a3e4e0fe33aee4b1
Author: Stefano Lattarini <address@hidden>
Date:   Wed Jul 27 15:33:13 2011 +0200

    tests: fix spurious failures with Solaris /bin/sh
    
    * tests/tap-more.test: Use `echo > file', not `: > file', to
    create empty files in the "for" loops; this is required since,
    as documented in the autoconf manual, Solaris 10 /bin/sh
    "optimizes" away the `:' command after the first iteration,
    even if it is redirected.
    * tests/test-driver-custom-multitest-recheck2.test: Likewise.
    * tests/tap-numeric-description.test: Partial rewrite to avoid
    using positional parameters from the 10th onward, which are
    unportable to Solaris /bin/sh (using `${10}' causes the shell
    to die with "bad substitution").

commit c04e7173208bb56d094e676c603f42e274ae1a3d
Author: Stefano Lattarini <address@hidden>
Date:   Sun Jul 24 16:47:24 2011 +0200

    simple tests: support developer-defined fd redirections
    
    Motivated by coreutils bug#8846, and related discussions:
     <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=8846>
     <http://lists.gnu.org/archive/html/bug-autoconf/2011-06/msg00002.html>
    In those threads it has been shown how problematic it can be to do
    portable file descriptor redirections in the testsuite when using
    the Automake testsuite harness.  This change should remedy to that
    situation.
    
    * lib/am/check2.am (?GENERIC?%EXT%$(EXEEXT).log,
    ?GENERIC?%EXT%.log, ?!GENERIC?%OBJ%): Append redirection defined
    in $(AM_TESTS_FD_REDIRECT) to the command-line invocations of the
    test scripts.
    * lib/am/check.am [!%?PARALLEL_TESTS%] $(check-TESTS): Likewise.
    * NEWS: Update.
    * doc/automake.texi (Script-based Testsuites): Document the new
    feature.
    * tests/check-fd-redirect.test: New test.
    * tests/parallel-tests-fd-redirect.test: Likewise.
    * tests/parallel-tests-am_tests_environment.test: Remove checks
    about the use of redirections in AM_TESTS_ENVIRONMENT: they would
    check deprecated (if not undefined) behaviour now.  Strengthen a
    couple of still valid checks, to keep the test more in sync with
    the documentation.  Improve debugging information.
    * tests/Makefile.am (TESTS): Update.

commit 382d084e5754c8f7fc7b622704ead916ff30d40b
Author: Stefano Lattarini <address@hidden>
Date:   Thu Jul 21 17:07:54 2011 +0200

    docs: rework and extend documentation on testsuites support
    
    * doc/automake.texi (Generalities about Testing): New section.
    (Simple Tests using parallel-tests): Section removed, merged
    into ...
    (Simple Tests): ... this one, which on the other hand has been
    subdivided into ...
    (Script-based Testsuites, Serial Test Harness, Parallel Test
    Harness): ... these new subsection.
    (DejaGnu Tests): Minor adjustments.
    Other related typofixes and rewordings throughout the manual;
    in particular, avoid to use the term "test driver" for three
    different concepts (and use instead "test harness" and "test
    runner" where appropriate).
    * tests/tap-doc.test: New test.
    * tests/tap-no-disable-hard-error.test: Likewise.
    * tests/Makefile.am (TESTS): Update.

commit 415cd5bbc9ba07186db473540e0ac3d03dffb2fb
Author: Stefano Lattarini <address@hidden>
Date:   Sun Jul 17 16:46:41 2011 +0200

    tap: support colorization of testsuite progress output
    
    * lib/tap-driver (%COLORS): New variable (definition extracted
    from `lib/am/check.am:$(am__tty_colors)', with some obvious
    adjustments.
    (report): Adjust to colorize console output when required,
    using ...
    (decorate_result): ... this new function.
    (colored): New function, used by the one above.
    * tests/tap-summary.test: Also run the checks when `color-tests'
    is in use.
    * tests/Makefile.am (XFAIL_TESTS): Remove `tap-color.test'.

commit 38d225c86ab1dbcf28eecb3fb05501ca9e0430c7
Author: Stefano Lattarini <address@hidden>
Date:   Sun Jul 17 16:04:06 2011 +0200

    tap: some preparatory refactoring (2)
    
    This is a follow-up simplification.
    
    * lib/tap-driver (console_output): Renamed ...
    (report): ... to this, and extended to appropriately register
    the test results when needed.
    (testsuite_error, handle_tap_comment, handle_tap_test,
    handle_tap_plan): Adjusted accordingly.

commit 43b4a92e3dc8b2dafc9b828d590954aba379be5b
Author: Stefano Lattarini <address@hidden>
Date:   Sun Jul 17 15:18:51 2011 +0200

    tap: some preparatory refactoring (1)
    
    This refactoring is aimed at simplifying the introduction of
    colored console output for the TAP driver.
    
    * lib/tap-driver (console_output): Now accept two arguments, the
    first one indicating which kind of thing is to be displayed (for
    now only a test result or a diagnostic comment), and the second
    one (if present) the message associated to it.
    (handle_tap_test, handle_tap_comment, handle_tap_plan,
    testsuite_error): Adapt to the new `console_output' interface.

commit c761822446ef1421ccc57c902b00cb5f804f7dd9
Author: Stefano Lattarini <address@hidden>
Date:   Sun Jul 3 19:20:22 2011 +0200

    tap: add experimental TAP-aware driver
    
    * doc/automake.texi (Using the TAP test protocol): New section.
    (Overview of Custom Test Drivers Support): Minor updates.
    * lib/tap-driver: New script, TAP-aware test driver for Automake;
    implemented in perl and based on TAP::Parser.
    * lib/Makefile.am (dist_script_DATA): Add it.
    * tests/tap-autonumber.test: New test.
    * tests/tap-bailout.test: Likewise.
    * tests/tap-basic.test: Likewise.
    * tests/tap-deps.test: Likewise.
    * tests/tap-diagnostic.test: Likewise.
    * tests/tap-empty.test: Likewise.
    * tests/tap-empty-diagnostic.test: Likewise.
    * tests/tap-escape-directive.test: Likewise.
    * tests/tap-exit.test: Likewise.
    * tests/tap-fancy.test: Likewise.
    * tests/tap-fancy2.test: Likewise.
    * tests/tap-global-result.test: Likewise.
    * tests/tap-html.test: Likewise.
    * tests/tap-log.test: Likewise.
    * tests/tap-merge-stdout-stderr.test: Likewise.
    * tests/tap-more.test: Likewise.
    * tests/tap-more2.test: Likewise.
    * tests/tap-no-merge-stdout-stderr.test: Likewise.
    * tests/tap-no-spurious-summary.test: Likewise.
    * tests/tap-no-spurious.test: Likewise.
    * tests/tap-not-ok-skip.test: Likewise.
    * tests/tap-numeric-description.test: Likewise.
    * tests/tap-out-of-order.test: Likewise.
    * tests/tap-passthrough.test: Likewise.
    * tests/tap-plan.test: Likewise.
    * tests/tap-plan-errors.test: Likewise.
    * tests/tap-plan-corner.test: Likewise.
    * tests/tap-realtime.test: Likewise.
    * tests/tap-recheck-logs.test: Likewise.
    * tests/tap-recheck.test: Likewise.
    * tests/tap-skip-whole.test: Likewise.
    * tests/tap-summary.test: Likewise.
    * tests/tap-todo-skip.test: Likewise.
    * tests/tap-todo-skip-together.test: Likewise.
    * tests/tap-todo-skip-whitespace.test: Likewise.
    * tests/tap-skipall-whitespace.test: Likewise.
    * tests/tap-unplanned.test: Likewise.
    * tests/tap-whitespace-normalization.test: Likewise.
    * tests/tap-with-and-without-number.test: Likewise.
    * tests/tap-xfail-tests.test: Likewise.
    * tests/tap-bad-prog.test: New xfailing test.
    * tests/tap-color.test: Likewise.
    * tests/tap-plan-corner2.test: Likewise.
    * tests/tap-message-0.test: Likewise.
    * tests/tap-signal.test: Likewise.
    * tests/Makefile.am (TESTS, XFAIL_TESTS): Update.

commit 505746496d1ff30d423d3759cdfd1792d1be3420
Author: Stefano Lattarini <address@hidden>
Date:   Mon Jul 4 18:12:07 2011 +0200

    tests defs: new auxiliary function 'count_test_results'
    
    * tests/defs (count_test_results): New function.
    * tests/check11.test: Use it.
    * tests/test-driver-custom-multitest.test: Likewise.
    * tests/test-driver-custom-multitest-recheck.test: Likewise.
    * tests/test-driver-custom-multitest-recheck2.test: Likewise.
    * tests/parallel-tests-log-override-recheck.test: Likewise.
    * tests/parallel-tests-log-override-recheck.test: Likewise.
    * tests/parallel-tests-no-spurious-summary.test: Likewise, and
    slightly improve debugging output.
    * tests/parallel-tests.test: Make use of `count_test_results'.
    Also, make grepping of "make check" output slightly stricter
    * tests/parallel-tests9.test: Likewise.
    * tests/parallel-tests-log-override-2.test: Likewise, and throw
    in a small optimization.

commit eaac33fe13811cf104f45a09a6ece911c135fa83
Author: Stefano Lattarini <address@hidden>
Date:   Fri Jul 1 11:12:31 2011 +0200

    parallel-tests: simplify testsuite summary
    
    Prefer a more deterministic, "tabular" format for the testsuite
    summary, always listing the numbers of passed, failed, xfailed,
    xpassed, skipped and errored tests, even when these numbers are
    zero.  This simplify the logic of testsuite summary creation,
    makes it more easily machine-parseable, and will probably allow
    for easier addition of new kinds of test results in the future.
    
    * lib/am/check.am (am__tty_colors_dummy): New make variable, to
    reduce code duplication.  Extracted from previous versions of
    $(am__tty_colors), and extended by defining two new variables
    `$mgn' and `$brg'.
    [%?COLOR%, %!?COLOR%] (am__tty_colors): Use that new variable.
    (am__text_box): Delete, is not needed anymore.
    ($(TEST_SUITE_LOG)): Rewrite associated rules to implement the
    new testsuite summary format.
    * NEWS: Update.
    * tests/check10.test: Don't run with the parallel-tests harness
    too, that makes no sense anymore.
    * tests/color.test: Update and adjust.
    * tests/color2.test: Likewise.
    * tests/parallel-tests.test: Likewise.
    * tests/parallel-tests3.test: Likewise.
    * tests/parallel-tests6.test: Likewise.
    * tests/parallel-tests9.test: Likewise.
    * tests/parallel-tests-unreadable-log.test: Likewise.
    * tests/parallel-tests-empty-testlogs.test: Likewise.
    * tests/parallel-tests-log-override-recheck.test: Likewise.
    * tests/parallel-tests-no-spurious-summary.test: Likewise.
    * tests/test-driver-custom-multitest.test: Likewise.
    * tests/test-driver-end-test-results.test: Likewise.
    * tests/parallel-tests-no-color-in-log.test: New test.
    * tests/testsuite-summary-color.test: Likewise.
    * tests/testsuite-summary-count.test: Likewise.
    * tests/testsuite-summary-count-many.test: Likewise.
    * tests/testsuite-summary-reference-log.test: Likewise.
    * tests/testsuite-summary-checks.sh: New auxiliary script, used
    by the new tests above.
    * tests/extract-testsuite-summary: Likewise.
    * tests/trivial-test-driver: Optimize for speed when there are
    lots of of tests.
    * tests/Makefile.am (EXTRA_DIST): Distribute them.
    (testsuite-summary-color.log, testsuite-summary-count.log): Depend
    on them.
    (testsuite-summary-count-many.log): Depend on the auxiliary scripts
    'trivial-test-driver' and 'extract-testsuite-summary'.
    (TESTS): Update.

commit 15400bca303910d059f1a68f94b25cffb1a8ac2f
Author: Stefano Lattarini <address@hidden>
Date:   Thu Jun 30 14:55:43 2011 +0200

    parallel-tests: new recognized test result 'ERROR'
    
    * lib/am/check.am ($(TEST_SUITE_LOG)): Recognize a new test result
    `ERROR'.  Use it when encountering unreadable test logs (previously
    a simple `FAIL' was used in this situations).
    * lib/test-driver: Set the global test result to `ERROR' when the
    test exit status is 99.  When doing colorized output, color `ERROR'
    results in magenta.
    * doc/automake.texi (Log files generation and test results
    recording): Update by also listing `ERROR' among the list of valid
    `:test-results:' arguments.
    * NEWS: Update.
    * tests/trivial-test-driver: Update.
    * tests/parallel-tests.test: Likewise.
    * tests/parallel-tests-harderror.test: Likewise.
    * tests/parallel-tests-no-spurious-summary.test: Likewise.
    * tests/test-driver-global-log.test: Likewise.
    * tests/test-driver-recheck.test: Likewise.
    * tests/test-driver-custom-multitest-recheck.test: Likewise.
    * tests/test-driver-custom-multitest-recheck2.test: Likewise.
    * tests/test-driver-custom-multitest.test: Likewise.
    * tests/test-driver-custom-no-html.test: Likewise.
    * tests/test-driver-end-test-results.test: Likewise.
    * tests/color.test: Likewise.  Also, make stricter, and also test
    from VPATH.
    * tests/color2.test: Likewise, and improve syncing with color.test.
    * tests/parallel-tests-exit-statuses.test: New test.
    * tests/parallel-tests-console-output.test: Likewise.
    * tests/Makefile.am (TESTS): Update.

commit 61b2bb0d647ea5f192e49f5f5885009ea83968be
Author: Stefano Lattarini <address@hidden>
Date:   Wed Jun 29 16:02:52 2011 +0200

    parallel-tests: make parsing of test results safer
    
    The new code for parsing the testsuite-generated `.log' files,
    as introduced in commit `v1.11-872-gc96b881', considers each
    `:test-result:' field anywhere in a `.log' file as a declaration
    of a test result, and accounts for it as such in the testsuite
    summary.  Unfortunately this could easily cause spurious test
    failures being reported in the testsuite summary.  This happened
    in practice with the Automake's own testsuite; for example:
    
      $ make check TESTS='check12-p.test'; echo exit: $?
      ...
      PASS: check12-p.test
      =====================================
      4 of 5 tests failed
      See tests/test-suite.log
      Please report to address@hidden
      =====================================
      make[2]: *** [test-suite.log] Error 1
      make: *** [check-am] Error 2
      exit: 2
    
    This change introduces a new special `:test-result:' "END", that,
    when seen, prevents the rest of the log file from being parsed.
    
    For more information, refer to the thread:
    <http://lists.gnu.org/archive/html/automake-patches/2011-06/msg00199.html>
    
    * lib/am/check.am ($(TEST_SUITE_LOG)): Stop the parsing of a log
    file as soon as the special ":test-result:END" directive is seen.
    Related changes and enhancements.
    * lib/test-driver: Protect the rest of the log after the result
    lined with a ":test-result:END" directive.
    * tests/parallel-tests-no-spurious-summary.test: New test.
    * tests/test-driver-end-test-results.test: Likewise.
    * tests/Makefile.am (TESTS): Update.

-----------------------------------------------------------------------


hooks/post-receive
-- 
GNU Automake