bug#14760: GNU Automake 1.14 testsuite failures on -current NetBSD/amd64

[Nicolas Joly]

On Mon, Jul 22, 2013 at 12:08:44AM +0100, Stefano Lattarini wrote:
> On 07/01/2013 04:05 PM, Nicolas Joly wrote:
> > 
> > Hi,
> > 
> > Just got 2 failures running Automake 1.14 testsuite on -current
> > NetBSD/amd64.
> >
> > [SNIP]
> >
> > FAIL: t/parallel-tests-concurrency
> > ==================================
> >
> > Running from installcheck: no
> > Test Protocol: none
> > PATH = 
> > /local/src/automake-1.14/t/wrap:/local/src/automake-1.14/t/ax:/home/njoly/bin:/local/bin:/local/sbin:/usr/pkg/bin:/usr/pkg/sbin:/usr/X11R7/bin:/usr/bin:/usr/sbin:/bin:/sbin
> > ++ pwd
> > /local/src/automake-1.14/t/parallel-tests-concurrency.dir
> > + case $MAKE in
> > + using_gmake
> > + case $am__using_gmake in
> > + grep GNU
> > + make --version -v
> > usage: make [-BeikNnqrstWX]
> >             [-C directory] [-D variable] [-d flags] [-f makefile]
> >             [-I directory] [-J private] [-j max_jobs] [-m directory] [-T 
> > file]
> >             [-V variable] [variable=value] [target ...]
> > + am__using_gmake=no
> > + return 1
> > + unindent
> > + test x = x
> > ++ sed -e 's/  *# .*//'
> > ++ printf '%s\n' '
> >   /^$/b                    # Nothing to do for empty lines.
> >   x                        # Get x<indent> into pattern space.
> >   /^$/{                    # No prior x<indent>, go prepare it.
> >     g                      # Copy this 1st non-blank line into pattern 
> > space.
> >     s/^\([   ]*\).*/x\1/   # Prepare x<indent> in pattern space.
> >   }                        # Now: x<indent> in pattern and <line> in hold.
> >   G                        # Build x<indent>\n<line> in pattern space, and
> >   h                        # duplicate it into hold space.
> >   s/\n.*$//                # Restore x<indent> in pattern space, and
> >   x                        # exchange with the above duplicate in hold 
> > space.
> >   s/^x\(.*\)\n\1//         # Remove leading <indent> from <line>.
> >   s/^x.*\n//               # Restore <line> when there is no leading 
> > <indent>.
> > '
> > + sed_unindent_prog='
> >   /^$/b
> >   x
> >   /^$/{
> >     g
> >     s/^\([   ]*\).*/x\1/
> >   }
> >   G
> >   h
> >   s/\n.*$//
> >   x
> >   s/^x\(.*\)\n\1//
> >   s/^x.*\n//'
> > + sed '
> >   /^$/b
> >   x
> >   /^$/{
> >     g
> >     s/^\([   ]*\).*/x\1/
> >   }
> >   G
> >   h
> >   s/\n.*$//
> >   x
> >   s/^x\(.*\)\n\1//
> >   s/^x.*\n//'
> > + for j in '"-j"' '"-j "' NONE
> > + test x-j = xNONE
> > + run_make -M -- -j2 all
> > + am__make_redirect_stdout=no
> > + am__make_redirect_stderr=no
> > + am__make_redirect_stdall=no
> > + am__make_flags=
> > + am__make_rc_exp=0
> > + am_make_rc=0
> > + test 4 -gt 0
> > + case $1 in
> > + am__make_redirect_stdall=yes
> > + shift
> > + test 3 -gt 0
> > + case $1 in
> > + shift
> > + break
> > + using_gmake
> > + case $am__using_gmake in
> > + return 1
> > + for am__x in '"$@"'
> > + case $am__x in
> > + for am__x in '"$@"'
> > + case $am__x in
> > + unset am__x
> > + test x '!=' x
> > + : exec make -j2 all
> > + set +x
> > + test 0 -eq 253
> > + test yes = yes
> > + cat output
> > --- one ---
> > --- two ---
> > echo ok > two
> > --- one ---
> > sleep 2 && test -f two
> > + case $am__make_rc_exp in
> > + test 0 -ge 0
> > + test 0 -le 255
> > + test 0 -eq 0
> > + /usr/bin/grep -E -i '(warning|error):|-j["\'\''` ]' output
> > + break
> > + rm -f one output Makefile
> > + cat
> > + cat
> > + cat
> > + chmod a+x ./x
> > + mkdir sub
> > + for i in 1 2 3
> > + echo 'TESTS += foo1.test'
> > + cp x foo1.test
> > + echo 'TESTS += zap1'
> > + cp x zap1
> > + echo 'TESTS += sub/bar1.test'
> > + cp x sub/bar1.test
> > + echo 'TESTS += sub/mu1'
> > + cp x sub/mu1
> > + for i in 1 2 3
> > + echo 'TESTS += foo2.test'
> > + cp x foo2.test
> > + echo 'TESTS += zap2'
> > + cp x zap2
> > + echo 'TESTS += sub/bar2.test'
> > + cp x sub/bar2.test
> > + echo 'TESTS += sub/mu2'
> > + cp x sub/mu2
> > + for i in 1 2 3
> > + echo 'TESTS += foo3.test'
> > + cp x foo3.test
> > + echo 'TESTS += zap3'
> > + cp x zap3
> > + echo 'TESTS += sub/bar3.test'
> > + cp x sub/bar3.test
> > + echo 'TESTS += sub/mu3'
> > + cp x sub/mu3
> > + aclocal-1.14 -Werror
> > + autoconf
> > + automake-1.14 --foreign -Werror -Wall -a
> > parallel-tests: installing './test-driver'
> > + for build in serial parallel
> > + mkdir serial
> > + cd serial
> > + ../configure
> > checking for a BSD-compatible install... /usr/bin/install -c
> > checking whether build environment is sane... yes
> > checking for a thread-safe mkdir -p... ../install-sh -c -d
> > checking for gawk... no
> > checking for mawk... no
> > checking for nawk... no
> > checking for awk... awk
> > checking whether make sets $(MAKE)... yes
> > checking whether make supports nested variables... yes
> > checking that generated files are newer than configure... done
> > configure: creating ./config.status
> > config.status: creating Makefile
> > + make
> > + cd ..
> > + for build in serial parallel
> > + mkdir parallel
> > + cd parallel
> > + ../configure
> > checking for a BSD-compatible install... /usr/bin/install -c
> > checking whether build environment is sane... yes
> > checking for a thread-safe mkdir -p... ../install-sh -c -d
> > checking for gawk... no
> > checking for mawk... no
> > checking for nawk... no
> > checking for awk... awk
> > checking whether make sets $(MAKE)... yes
> > checking whether make supports nested variables... yes
> > checking that generated files are newer than configure... done
> > configure: creating ./config.status
> > config.status: creating Makefile
> > + make
> > + cd ..
> > + cd serial
> > + cd ../parallel
> > + sleep 2
> > + make -j1 check
> > make  check-TESTS
> > PASS: foo1.test
> > + run_make -O -- -j4 check
> > + am__make_redirect_stdout=no
> > + am__make_redirect_stderr=no
> > + am__make_redirect_stdall=no
> > + am__make_flags=
> > + am__make_rc_exp=0
> > + am_make_rc=0
> > + test 4 -gt 0
> > + case $1 in
> > + am__make_redirect_stdout=yes
> > + shift
> > + test 3 -gt 0
> > + case $1 in
> > + shift
> > + break
> > + using_gmake
> > + case $am__using_gmake in
> > + return 1
> > + for am__x in '"$@"'
> > + case $am__x in
> > + for am__x in '"$@"'
> > + case $am__x in
> > + unset am__x
> > + test x '!=' x
> > + : exec make -j4 check
> > + set +x
> > PASS: zap1
> > PASS: sub/bar1.test
> > PASS: sub/mu1
> > PASS: foo2.test
> > PASS: zap2
> > PASS: sub/bar2.test
> > PASS: sub/mu2
> > PASS: foo3.test
> > PASS: zap3
> > PASS: sub/bar3.test
> > PASS: sub/mu3
> > ============================================================================
> > Testsuite summary for parallel-tests-concurrency 1.0
> > ============================================================================
> > # TOTAL: 12
> > # PASS:  12
> > # SKIP:  0
> > # XFAIL: 0
> > # FAIL:  0
> > # XPASS: 0
> > # ERROR: 0
> > ============================================================================
> > + test 0 -eq 253
> > + test no = yes
> > + test yes = yes
> > + cat stdout
> > --- check-am ---
> > make  check-TESTS
> > --- check-TESTS ---
> > PASS: foo1.test
> > PASS: zap1
> > PASS: sub/bar1.test
> > PASS: sub/mu1
> > PASS: foo2.test
> > PASS: zap2
> > PASS: sub/bar2.test
> > PASS: sub/mu2
> > PASS: foo3.test
> > PASS: zap3
> > PASS: sub/bar3.test
> > PASS: sub/mu3
> > ============================================================================
> > Testsuite summary for parallel-tests-concurrency 1.0
> > ============================================================================
> > # TOTAL: 12
> > # PASS:  12
> > # SKIP:  0
> > # XFAIL: 0
> > # FAIL:  0
> > # XPASS: 0
> > # ERROR: 0
> > ============================================================================
> > + test no = yes
> > + case $am__make_rc_exp in
> > + test 0 -ge 0
> > + test 0 -le 255
> > + test 0 -eq 0
> > + cd ..
> > + kill -0 23026
> > ./t/parallel-tests-concurrency.sh: line 100: kill: (23026) - No such process
> > + am_exit_trap 1
> > + exit_status=1
> > + set +e
> > + cd /local/src/automake-1.14
> > + test none = tap
> > + case $am_explicit_skips in
> > + test 1 -eq 0
> > + keep_testdirs=yes
> > + am_keeping_testdirs
> > + case $keep_testdirs in
> > + return 0
> > + set +x
> > parallel-tests-concurrency: exit 1
> >
> I must admit I currently have no idea about what might be the root
> cause of this failure.  So, few questions to try to shed more light
> on the issue:
> 
>   1. Is the failure consistently reproducible?

Yes.

address@hidden [src/automake-1.14]> make -k recheck
FAIL: t/parallel-tests-concurrency.sh
============================================================================
Testsuite summary for GNU Automake 1.14
============================================================================

>   2. Is the failure present only when you run the test from make,
>      or also when you run it "directly" (that is, through a
>      ./runtest invocation)?

Same failure with runtest :

address@hidden [src/automake-1.14]> ./runtest t/parallel-tests-concurrency.sh
[...]
+ kill -0 5934
./t/parallel-tests-concurrency.sh: line 100: kill: (5934) - No such process
+ am_exit_trap 1
+ exit_status=1
+ set +e
[...]
+ case $keep_testdirs in
+ return 0
+ set +x
parallel-tests-concurrency: exit 1

>   3. If the failure is racy, does it happen more often under a
>      loaded system and/or when the Automake testsuite is run in
>      low priority, or is that irrelevant?

Doesn't seems relevant.

>   4. Any other hypothesis/suggestion you want to share?

Seems to be $MAKE related ... The test succeed with gmake, but fail
with NetBSD make. Both logs attached for review.

Thanks.

-- 
Nicolas Joly

Biology IT Center
Institut Pasteur, Paris.

gmake.log
Description: Text document

make.log
Description: Text document