Re: testsuite failures when test scripts are run with zsh

[Stefano Lattarini]

Stefano Lattarini <address@hidden> wrote:
> Hi everybody.
>
> I forced the automake testsuite to run tests in parallel using the
> *Zsh* shell, with the following command:
>  $ make -j5 TEST_LOG_COMPILER='zsh  --nofunctionargzero' check
>
> The following failures turned up:
>
> FAIL: aclocal8.test (exit: 1)
> FAIL: acsilent.test (exit: 1)
> FAIL: depcomp2.test (exit: 1)
> FAIL: getopt.test (exit: 1)
> FAIL: location.test (exit: 1)
> FAIL: missing3.test (exit: 1)
> FAIL: output-order.test (exit: 1)
> FAIL: parallel-tests4.test (exit: 2)
> FAIL: unused.test (exit: 1)
> FAIL: warnopts.test (exit: 1)
> FAIL: check9-p.test (exit: 2)
>
> [CUT]
>
> Regards,
>    Stefano
If anybody is still interested in this report: I did some further
investigations, and it soon turned out that the failures are due to
the fact that zsh (version 4 or later) handle the `-x' option in a
totally brain-damaged way. In details: if a command which is run
when `set -x' is active has its standard error redirected to a file,
then zsh writes *also the trace* for the command into that file,
instead that into the original standard error.

The following examples should help making things clearer (warning:
I tried these only on my system, thus YMMV):
 
 $ bash -x -c 'echo bash >out 2>&1'
 + echo bash
 $ cat out
 bash
 
 $ dash -x -c 'echo dash >out 2>&1'
 + echo dash
 $ cat out
 dash
 
 $ ksh -x -c 'echo ksh >out 2>&1'
 + echo ksh
 + 1> out 2>& 1
 $ cat out
 ksh
 
 $ zsh3 -x -c 'echo zsh3 >out 2>&1'
 + echo zsh3
 $ cat out
 zsh3
 
 $ zsh4 -x -c 'echo zsh4 >out 2>&1'
 $ cat out
 +zsh:1> echo zsh
 zsh4

I found no workaround.  The only way I can think of to fix these
spurious failures involves modifying all the failing scripts in a 
hackish way, to work around this stupid behaviour of zsh.

IMHO, the best thing to do at this point is symply to edit the file 
`tests/README' and make it clearly state that Zsh (version 4 or 
later) is not supported by the testsuite.
By the way, the only systems I heard of that use zsh as /bin/sh
were MacOS X 10.0 or 10.1, and goolge tells me that they used
a 3.0.something version of zsh, which does not show the
aforementioned bug.

Regards,
   Stefano