bug#14760: GNU Automake 1.14 testsuite failures on -current NetBSD/amd64

[Stefano Lattarini]

On 07/01/2013 04:05 PM, Nicolas Joly wrote:
> 
> Hi,
> 
> Just got 2 failures running Automake 1.14 testsuite on -current
> NetBSD/amd64.
>
> [SNIP]
>
> FAIL: t/parallel-tests-concurrency
> ==================================
>
> Running from installcheck: no
> Test Protocol: none
> PATH = 
> /local/src/automake-1.14/t/wrap:/local/src/automake-1.14/t/ax:/home/njoly/bin:/local/bin:/local/sbin:/usr/pkg/bin:/usr/pkg/sbin:/usr/X11R7/bin:/usr/bin:/usr/sbin:/bin:/sbin
> ++ pwd
> /local/src/automake-1.14/t/parallel-tests-concurrency.dir
> + case $MAKE in
> + using_gmake
> + case $am__using_gmake in
> + grep GNU
> + make --version -v
> usage: make [-BeikNnqrstWX]
>             [-C directory] [-D variable] [-d flags] [-f makefile]
>             [-I directory] [-J private] [-j max_jobs] [-m directory] [-T file]
>             [-V variable] [variable=value] [target ...]
> + am__using_gmake=no
> + return 1
> + unindent
> + test x = x
> ++ sed -e 's/  *# .*//'
> ++ printf '%s\n' '
>   /^$/b                    # Nothing to do for empty lines.
>   x                        # Get x<indent> into pattern space.
>   /^$/{                    # No prior x<indent>, go prepare it.
>     g                      # Copy this 1st non-blank line into pattern space.
>     s/^\([     ]*\).*/x\1/   # Prepare x<indent> in pattern space.
>   }                        # Now: x<indent> in pattern and <line> in hold.
>   G                        # Build x<indent>\n<line> in pattern space, and
>   h                        # duplicate it into hold space.
>   s/\n.*$//                # Restore x<indent> in pattern space, and
>   x                        # exchange with the above duplicate in hold space.
>   s/^x\(.*\)\n\1//         # Remove leading <indent> from <line>.
>   s/^x.*\n//               # Restore <line> when there is no leading <indent>.
> '
> + sed_unindent_prog='
>   /^$/b
>   x
>   /^$/{
>     g
>     s/^\([     ]*\).*/x\1/
>   }
>   G
>   h
>   s/\n.*$//
>   x
>   s/^x\(.*\)\n\1//
>   s/^x.*\n//'
> + sed '
>   /^$/b
>   x
>   /^$/{
>     g
>     s/^\([     ]*\).*/x\1/
>   }
>   G
>   h
>   s/\n.*$//
>   x
>   s/^x\(.*\)\n\1//
>   s/^x.*\n//'
> + for j in '"-j"' '"-j "' NONE
> + test x-j = xNONE
> + run_make -M -- -j2 all
> + am__make_redirect_stdout=no
> + am__make_redirect_stderr=no
> + am__make_redirect_stdall=no
> + am__make_flags=
> + am__make_rc_exp=0
> + am_make_rc=0
> + test 4 -gt 0
> + case $1 in
> + am__make_redirect_stdall=yes
> + shift
> + test 3 -gt 0
> + case $1 in
> + shift
> + break
> + using_gmake
> + case $am__using_gmake in
> + return 1
> + for am__x in '"$@"'
> + case $am__x in
> + for am__x in '"$@"'
> + case $am__x in
> + unset am__x
> + test x '!=' x
> + : exec make -j2 all
> + set +x
> + test 0 -eq 253
> + test yes = yes
> + cat output
> --- one ---
> --- two ---
> echo ok > two
> --- one ---
> sleep 2 && test -f two
> + case $am__make_rc_exp in
> + test 0 -ge 0
> + test 0 -le 255
> + test 0 -eq 0
> + /usr/bin/grep -E -i '(warning|error):|-j["\'\''` ]' output
> + break
> + rm -f one output Makefile
> + cat
> + cat
> + cat
> + chmod a+x ./x
> + mkdir sub
> + for i in 1 2 3
> + echo 'TESTS += foo1.test'
> + cp x foo1.test
> + echo 'TESTS += zap1'
> + cp x zap1
> + echo 'TESTS += sub/bar1.test'
> + cp x sub/bar1.test
> + echo 'TESTS += sub/mu1'
> + cp x sub/mu1
> + for i in 1 2 3
> + echo 'TESTS += foo2.test'
> + cp x foo2.test
> + echo 'TESTS += zap2'
> + cp x zap2
> + echo 'TESTS += sub/bar2.test'
> + cp x sub/bar2.test
> + echo 'TESTS += sub/mu2'
> + cp x sub/mu2
> + for i in 1 2 3
> + echo 'TESTS += foo3.test'
> + cp x foo3.test
> + echo 'TESTS += zap3'
> + cp x zap3
> + echo 'TESTS += sub/bar3.test'
> + cp x sub/bar3.test
> + echo 'TESTS += sub/mu3'
> + cp x sub/mu3
> + aclocal-1.14 -Werror
> + autoconf
> + automake-1.14 --foreign -Werror -Wall -a
> parallel-tests: installing './test-driver'
> + for build in serial parallel
> + mkdir serial
> + cd serial
> + ../configure
> checking for a BSD-compatible install... /usr/bin/install -c
> checking whether build environment is sane... yes
> checking for a thread-safe mkdir -p... ../install-sh -c -d
> checking for gawk... no
> checking for mawk... no
> checking for nawk... no
> checking for awk... awk
> checking whether make sets $(MAKE)... yes
> checking whether make supports nested variables... yes
> checking that generated files are newer than configure... done
> configure: creating ./config.status
> config.status: creating Makefile
> + make
> + cd ..
> + for build in serial parallel
> + mkdir parallel
> + cd parallel
> + ../configure
> checking for a BSD-compatible install... /usr/bin/install -c
> checking whether build environment is sane... yes
> checking for a thread-safe mkdir -p... ../install-sh -c -d
> checking for gawk... no
> checking for mawk... no
> checking for nawk... no
> checking for awk... awk
> checking whether make sets $(MAKE)... yes
> checking whether make supports nested variables... yes
> checking that generated files are newer than configure... done
> configure: creating ./config.status
> config.status: creating Makefile
> + make
> + cd ..
> + cd serial
> + cd ../parallel
> + sleep 2
> + make -j1 check
> make  check-TESTS
> PASS: foo1.test
> + run_make -O -- -j4 check
> + am__make_redirect_stdout=no
> + am__make_redirect_stderr=no
> + am__make_redirect_stdall=no
> + am__make_flags=
> + am__make_rc_exp=0
> + am_make_rc=0
> + test 4 -gt 0
> + case $1 in
> + am__make_redirect_stdout=yes
> + shift
> + test 3 -gt 0
> + case $1 in
> + shift
> + break
> + using_gmake
> + case $am__using_gmake in
> + return 1
> + for am__x in '"$@"'
> + case $am__x in
> + for am__x in '"$@"'
> + case $am__x in
> + unset am__x
> + test x '!=' x
> + : exec make -j4 check
> + set +x
> PASS: zap1
> PASS: sub/bar1.test
> PASS: sub/mu1
> PASS: foo2.test
> PASS: zap2
> PASS: sub/bar2.test
> PASS: sub/mu2
> PASS: foo3.test
> PASS: zap3
> PASS: sub/bar3.test
> PASS: sub/mu3
> ============================================================================
> Testsuite summary for parallel-tests-concurrency 1.0
> ============================================================================
> # TOTAL: 12
> # PASS:  12
> # SKIP:  0
> # XFAIL: 0
> # FAIL:  0
> # XPASS: 0
> # ERROR: 0
> ============================================================================
> + test 0 -eq 253
> + test no = yes
> + test yes = yes
> + cat stdout
> --- check-am ---
> make  check-TESTS
> --- check-TESTS ---
> PASS: foo1.test
> PASS: zap1
> PASS: sub/bar1.test
> PASS: sub/mu1
> PASS: foo2.test
> PASS: zap2
> PASS: sub/bar2.test
> PASS: sub/mu2
> PASS: foo3.test
> PASS: zap3
> PASS: sub/bar3.test
> PASS: sub/mu3
> ============================================================================
> Testsuite summary for parallel-tests-concurrency 1.0
> ============================================================================
> # TOTAL: 12
> # PASS:  12
> # SKIP:  0
> # XFAIL: 0
> # FAIL:  0
> # XPASS: 0
> # ERROR: 0
> ============================================================================
> + test no = yes
> + case $am__make_rc_exp in
> + test 0 -ge 0
> + test 0 -le 255
> + test 0 -eq 0
> + cd ..
> + kill -0 23026
> ./t/parallel-tests-concurrency.sh: line 100: kill: (23026) - No such process
> + am_exit_trap 1
> + exit_status=1
> + set +e
> + cd /local/src/automake-1.14
> + test none = tap
> + case $am_explicit_skips in
> + test 1 -eq 0
> + keep_testdirs=yes
> + am_keeping_testdirs
> + case $keep_testdirs in
> + return 0
> + set +x
> parallel-tests-concurrency: exit 1
>
I must admit I currently have no idea about what might be the root
cause of this failure.  So, few questions to try to shed more light
on the issue:

  1. Is the failure consistently reproducible?

  2. Is the failure present only when you run the test from make,
     or also when you run it "directly" (that is, through a
     ./runtest invocation)?

  3. If the failure is racy, does it happen more often under a
     loaded system and/or when the Automake testsuite is run in
     low priority, or is that irrelevant?

  4. Any other hypothesis/suggestion you want to share?

Thanks,
  Stefano