Re: valgrind-tests.m4

[Simon Josefsson]

Bruno Haible <address@hidden> writes:

> The approach I use in GNU gettext is similar: Makefile.am has this:
>
> # For debugging memory leaks and memory allocation bugs.
> # You should build with --disable-shared when using valgrind.
> CHECKER =
> #CHECKER = valgrind --tool=memcheck 
> --suppressions=$(srcdir)/../gnulib-lib/malloca.valgrind --num-callers=20 
> --leak-check=yes --leak-resolution=high --show-reachable=yes
> #CHECKER = valgrind --tool=massif --format=html --depth=10 --alloc-fn=xmalloc 
> --alloc-fn=xrealloc --stacks=no
>
> TESTS_ENVIRONMENT = top_srcdir=$(top_srcdir) \
>                     
> PATH=.:../src:../../gettext-runtime/src:$(top_srcdir)/../gettext-runtime/src:$$PATH
>  \
>                     GETTEXT="$(CHECKER) tstgettext" \
>                     NGETTEXT="$(CHECKER) tstngettext" \
>                     XGETTEXT="$(CHECKER) xgettext" \
>                     MSGATTRIB="$(CHECKER) msgattrib" \
>                     MSGCAT="$(CHECKER) msgcat" \
>                     MSGCMP="$(CHECKER) msgcmp" \
>                     MSGCOMM="$(CHECKER) msgcomm" \
>                     MSGCONV="$(CHECKER) msgconv" \
>                     MSGEN="$(CHECKER) msgen" \
>                     MSGEXEC="$(CHECKER) msgexec" \
>                     MSGFILTER="$(CHECKER) msgfilter" \
>                     MSGFMT="$(CHECKER) msgfmt" \
>                     MSGGREP="$(CHECKER) msggrep" \
>                     MSGINIT="$(CHECKER) msginit" \
>                     MSGMERGE="$(CHECKER) msgmerge" \
>                     MSGUNFMT="$(CHECKER) msgunfmt" \
>                     MSGUNIQ="$(CHECKER) msguniq" \
>                     ...
>
> This idiom has the effect that:
>   - Use of valgrind is turned off by default.
>     I think that's the right setting, because
>       1. tests running with valgrind consume much more time that without.
>       2. some tests may fail due to bugs in valgrind. For example, valgrind
>          built for 32-bit failed miserably when run with 64-bit binaries on
>          x86 bi-arch systems.
>   - The user can enable it by changing 2 characters in the Makefile.
>   - The user does not have to remember the recommended options
>     (--alloc-fn=xmalloc etc.).
>   - The user has the choice among multiple valgrind tools.
>   - Shell invocations are not checked by valgrind, only the invocations
>     of the programs that are meant to be tested.

The second part (the TESTS_ENVIRONMENT rule) could easily be documented
in doc/valgrind-tests.texi, and I think that would be a useful
improvement.  I'll consider using this idiom as well, because it allows
mixing shell scripts and binaries more effectively.  I'm not sure how to
merge in your first idea (putting all the parameters in Makefile),
though, any ideas?  I'm not sure it is necessary -- the current patch
defines a variable VALGRIND that can be override by the user easily.

/Simon