bug#11512: Testsuite summary for GNU Automake 1.12

[Stefano Lattarini]

On 05/29/2012 12:35 AM, Guido Trentalancia wrote:
> 
> [SNIP]
>
> The above mentioned perl test failures were due to a broken perl
> installation. After re-installing perl, they complete successfully,
> so please ignore the above.
>
OK.

> The only test which fails is t/cond29.sh.
> 
>>> FAIL: t/cond29.sh
>>>
>>> FAIL: t/cond29
>>> ==============
> 
> It keeps failing unless the ulimit on virtual memory is increased to 500000 
> as you suggested...
> 
>>> Running from installcheck: no
>>> Using TAP: no
>>> PATH
>>> = 
>>> /usr/src/automake-1.12/t/ax:/usr/src/automake-1.12/t/wrap:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
>>> ++ pwd
>>> /usr/src/automake-1.12/t/cond29.dir
>>> + echo AC_PROG_CC
>>> + cat
>>> + ulimit -v 1
>>> + sh -c :
>>> ./t/cond29.sh: line 39:   613 Killed                  sh -c ":"
>>> + ulimit -v 150000
>>> + sh -c :
>>> + ulimit -v 150000
>>>
>>>  [SNIP]
>>>
>>> + echo 'AM_CONDITIONAL([C24], [:])'
>>> + aclocal-1.12 -Werror
>>> + automake-1.12 --foreign -Werror -Wall
>>> Out of memory!
>>> ./t/cond29.sh: line 61:   686 Segmentation fault      $AUTOMAKE
>>> + exit_status=139
>>>
>> What happens if you raise the ulimit above to (say) 500000?
> 
> The t/cond29.sh test completes successfully with ulimit -v 500000.
> 
> With 300000 and 200000 it still completes successfully... 
>
OK, so this is just a testsuite weakness, and not a "real" bug.  Good.

> So, the ulimit -v value needs to be increased from 150000 to at
> least 200000.
> 
> However, it's quite ugly to hard-code memory limits in the test code.
>
Quite true.  Maybe the best fix is to remove the 'ulimit' usages, and
make this tests (and the few other ones oriented at checking Automake
performances) only run if the user/developer explicitly ask for them,
say by exporting AM_RUN_PERF_TESTS to "yes".

> The default on my bash version 4.2 patchlevel 28 is unlimited virtual
> memory, can you not avoid limiting it, for example as follows:
> 
> --- automake-1.12/t/cond29.sh   2012-04-21 16:38:05.000000000 +0200
> +++ automake-1.12-work/t/cond29.sh      2012-05-29 00:24:10.248959036 +0200
> @@ -36,9 +36,6 @@ EOF
>  # that 70 MB is enough for x86_64 Fedora 16, that gives tools and
>  # libraries plenty of room to grow.  Suggested by Jim Meyering in
>  # automake bug#10374.
> -(ulimit -v 1; sh -c ":") && skip_ "no adequate 'ulimit' builtin found"
> -(ulimit -v 150000; sh -c ":") || skip_ "no adequate 'ulimit' builtin found"
> -ulimit -v 150000
>
The problem is that if we do so, then, in the unlikely possibility that the
bug this test case is checking against represents itself, the test will end
up crashing the tester's machine.  Not nice.

Thanks,
  Stefano