[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
gnulib-tool: support for unit test modules
|
From: |
Bruno Haible |
|
Subject: |
gnulib-tool: support for unit test modules |
|
Date: |
Thu, 25 Aug 2005 14:39:38 +0200 |
|
User-agent: |
KMail/1.5 |
Hi all,
C code generally doesn't work unless it's tested.
Simon added a test for the 'getpass' module yesterday, but no "make check"
actually runs it. So it resembles dead code. Let's change that!
Nelson Beebe has a zoo of different platforms, and now and then he reports
portability problems about a package that uses gnulib. But we have no
guarantee that all of gnulib actually works on all his platforms.
Let's change that!
I'm adding to gnulib-tool the support for unitary tests. It is activated
when you pass the option --with-tests. Then, the package created by
--create-testdir or --create-megatestdir has a "make check" that will
actually run the available tests.
When you contribute new tests, the test sources go under tests/.
They are accompanied with a metainformation file modules/<module>-tests.
The template for this file is:
Files:
[Here you list all test files. m4 files are also possible.]
Depends-on:
[The test module may depend on other modules than the particular library
module. It may even depend on other test modules.]
configure.ac:
[Some autoconf tests for use by the test.]
Makefile.am:
[Snippet of rules for tests/Makefile.am.]
Everyone, please contribute a test for your modules! Then at some point,
we can actually create a megatestdir and have people like Nelson Beebe
perform a complete portability test on gnulib.
Bruno
Support for unit test modules.
* modules/README: Mention tests modules.
* modules/TEMPLATE-TESTS: New file.
* gnulib-tool: New options --extract-tests-module, --with-tests and
--tests-base (unused for the moment).
(testsbase, inctests): New variables.
(func_all_modules): Exclude TEMPLATE-TESTS and *-tests.
(func_verify_module): Exclude TEMPLATE-TESTS.
(func_verify_nontests_module, func_verify_tests_module): New functions.
(func_get_dependencies): Add implicit dependency for tests modules.
(func_get_tests_module): New function.
(func_modules_transitive_closure): When --with-tests was specified,
include the unit tests as well, unless explicitly avoided.
(func_emit_lib_Makefile_am): Ignore the tests modules here.
(func_emit_tests_Makefile_am): New function.
(func_create_testdir): When --with-tests was specified, emit a
tests/ directory.
* MODULES.html.sh (Future developments): Update.
*** modules/README.bak 2004-10-11 13:28:34.000000000 +0200
--- modules/README 2005-08-25 00:54:40.000000000 +0200
***************
*** 1,9 ****
! This directory contains metainformation about the gnulib modules, one file
! per module. These files are used by gnulib-tool.
All the files in this directory are distributed under the following copyright:
! Copyright (C) 2002-2004 Free Software Foundation, Inc.
Copying and distribution of this file, with or without modification,
in any medium, are permitted without royalty provided the copyright
notice and this notice are preserved.
--- 1,15 ----
! This directory contains metainformation about the gnulib modules, one or two
! files per module. These files are used by gnulib-tool.
!
! For every module,
! - the file <module> is the metainformation about the library code of the
! module,
! - the file <module>-tests is the metainformation about the unit test of
! the module (optional but recommended).
All the files in this directory are distributed under the following copyright:
! Copyright (C) 2002-2005 Free Software Foundation, Inc.
Copying and distribution of this file, with or without modification,
in any medium, are permitted without royalty provided the copyright
notice and this notice are preserved.
*** gnulib-tool.bak 2005-08-25 02:14:54.000000000 +0200
--- gnulib-tool 2005-08-25 03:59:41.000000000 +0200
***************
*** 56,61 ****
--- 56,62 ----
gnulib-tool --extract-include-directive module
gnulib-tool --extract-license module
gnulib-tool --extract-maintainer module
+ gnulib-tool --extract-tests-module module
Operation modes:
--list print the available module names
***************
*** 76,81 ****
--- 77,83 ----
--extract-license report the license terms of the source
files
under lib/
--extract-maintainer report the maintainer(s) inside gnulib
+ --extract-tests-module report the unit test module, if it exists
Options:
--dir=DIRECTORY specify the target directory
***************
*** 88,95 ****
--- 90,101 ----
placed (default \"lib\"), for --import.
--m4-base=DIRECTORY Directory relative --dir where *.m4 macros are
placed (default \"m4\"), for --import.
+ --tests-base=DIRECTORY
+ Directory relative --dir where unit tests are
+ placed (default \"tests\"), for --import.
--aux-dir=DIRECTORY Directory relative --dir where auxiliary build
tools are placed (default \"build-aux\").
+ --with-tests Include unit tests for the included modules.
--avoid=MODULE Avoid including the given MODULE. Useful if you
have code that provides equivalent functionality.
This option can be repeated.
***************
*** 183,189 ****
--- 189,197 ----
# - libname, supplied_libname from --lib
# - sourcebase from --source-base
# - m4base from --m4-base
+ # - testsbase from --tests-base
# - auxdir from --aux-dir
+ # - inctests true if --with-tests was given, blank otherwise
# - avoidlist list of modules to avoid, from --avoid
# - lgpl true if --lgpl was given, blank otherwise
# - libtool true if --libtool was given, blank otherwise
***************
*** 196,202 ****
--- 204,212 ----
supplied_libname=
sourcebase=
m4base=
+ testsbase=
auxdir=
+ inctests=
avoidlist=
lgpl=
libtool=
***************
*** 271,276 ****
--- 281,296 ----
--m4-base=* )
m4base=`echo "X$1" | sed -e 's/^X--m4-base=//'`
shift ;;
+ --tests-base )
+ shift
+ if test $# = 0; then
+ func_fatal_error "missing argument for --tests-base"
+ fi
+ testsbase=$1
+ shift ;;
+ --tests-base=* )
+ testsbase=`echo "X$1" | sed -e 's/^X--tests-base=//'`
+ shift ;;
--aux-dir )
shift
if test $# = 0; then
***************
*** 281,286 ****
--- 301,309 ----
--aux-dir=* )
auxdir=`echo "X$1" | sed -e 's/^X--aux-dir=//'`
shift ;;
+ --with-tests )
+ inctests=true
+ shift ;;
--avoid )
shift
if test $# = 0; then
***************
*** 356,363 ****
# func_all_modules
func_all_modules ()
{
(cd "$gnulib_dir/modules" && ls -1) \
! | sed -e '/^CVS$/d' -e '/^ChangeLog$/d' -e '/^README$/d' -e
'/^TEMPLATE$/d' -e '/~$/d' \
| sort
}
--- 379,390 ----
# func_all_modules
func_all_modules ()
{
+ # Filter out metainformation files like README, which are not modules.
+ # Filter out unit test modules; they can be retrieved through
+ # --extract-tests-module if desired.
(cd "$gnulib_dir/modules" && ls -1) \
! | sed -e '/^CVS$/d' -e '/^ChangeLog$/d' -e '/^README$/d' -e
'/^TEMPLATE$/d' -e '/^TEMPLATE-TESTS$/d' -e '/~$/d' \
! | sed -e '/-tests$/d' \
| sort
}
***************
*** 369,380 ****
|| test "CVS" = "$module" \
|| test "ChangeLog" = "$module" \
|| test "README" = "$module" \
! || test "TEMPLATE" = "$module"; then
echo "gnulib-tool: module $module doesn't exist" 1>&2
module=
fi
}
sed_extract_prog=':[ ]*$/ {
:a
n
--- 396,428 ----
|| test "CVS" = "$module" \
|| test "ChangeLog" = "$module" \
|| test "README" = "$module" \
! || test "TEMPLATE" = "$module" \
! || test "TEMPLATE-TESTS" = "$module"; then
echo "gnulib-tool: module $module doesn't exist" 1>&2
module=
fi
}
+ # func_verify_nontests_module
+ # verifies a module name, excluding tests modules
+ func_verify_nontests_module ()
+ {
+ case "$module" in
+ *-tests ) module= ;;
+ * ) func_verify_module ;;
+ esac
+ }
+
+ # func_verify_tests_module
+ # verifies a module name, considering only tests modules
+ func_verify_tests_module ()
+ {
+ case "$module" in
+ *-tests ) func_verify_module ;;
+ * ) module= ;;
+ esac
+ }
+
sed_extract_prog=':[ ]*$/ {
:a
n
***************
*** 409,414 ****
--- 457,465 ----
# func_get_dependencies module
func_get_dependencies ()
{
+ # ${module}-tests always implicitly depends on ${module}.
+ echo "$1" | sed -n -e 's/-tests//p'
+ # Then the explicit dependencies listed in the module description.
sed -n -e "/^Depends-on$sed_extract_prog" < "$gnulib_dir/modules/$1"
}
***************
*** 443,448 ****
--- 494,508 ----
sed -n -e "/^Maintainer$sed_extract_prog" < "$gnulib_dir/modules/$1"
}
+ # func_get_tests_module module
+ func_get_tests_module ()
+ {
+ # The naming convention for tests modules is hardwired: ${module}-tests.
+ if test -f modules/"$1"-tests; then
+ echo "$1"-tests
+ fi
+ }
+
# func_acceptable module
# tests whether a module is acceptable.
# Input:
***************
*** 460,465 ****
--- 520,526 ----
# func_modules_transitive_closure
# Input:
# - modules list of specified modules
+ # - inctests true if tests should be included, blank otherwise
# - avoidlist list of modules to avoid
# Output:
# - modules list of modules, including dependencies
***************
*** 482,487 ****
--- 543,561 ----
xmodules="$xmodules $depmodule"
fi
done
+ if test -n "$inctests"; then
+ testsmodule=`func_get_tests_module $module`
+ if test -n "$testsmodule"; then
+ if func_acceptable $testsmodule; then
+ xmodules="$xmodules $testsmodule"
+ for depmodule in `func_get_dependencies $testsmodule`; do
+ if func_acceptable $depmodule; then
+ xmodules="$xmodules $depmodule"
+ fi
+ done
+ fi
+ fi
+ fi
fi
fi
done
***************
*** 560,566 ****
echo "MAINTAINERCLEANFILES ="
echo
for module in $modules; do
! func_verify_module
if test -n "$module"; then
{
func_get_automake_snippet "$module" |
--- 634,640 ----
echo "MAINTAINERCLEANFILES ="
echo
for module in $modules; do
! func_verify_nontests_module
if test -n "$module"; then
{
func_get_automake_snippet "$module" |
***************
*** 584,589 ****
--- 658,732 ----
echo "# Makefile.am ends here"
}
+ # func_emit_tests_Makefile_am
+ # emits the contents of tests/Makefile.am to standard output.
+ # Input:
+ # - modules list of modules, including dependencies
+ # - libname library name
+ # - libtool true if libtool will be used, blank otherwise
+ # - sourcebase relative directory containing lib source code
+ func_emit_tests_Makefile_am ()
+ {
+ if test -n "$libtool"; then
+ libext=la
+ else
+ libext=a
+ fi
+ echo "## Process this file with automake to produce Makefile.in."
+ echo "# Copyright (C) 2004-2005 Free Software Foundation, Inc."
+ echo "#"
+ echo "# This file is free software, distributed under the terms of the GNU"
+ echo "# General Public License. As a special exception to the GNU General"
+ echo "# Public License, this file may be distributed as part of a program"
+ echo "# that contains a configuration script generated by Automake, under"
+ echo "# the same distribution terms as the rest of that program."
+ echo "#"
+ echo "# Generated by gnulib-tool."
+ echo
+ # Generate dependencies here, since it eases the debugging of test failures.
+ echo "AUTOMAKE_OPTIONS = 1.5 foreign"
+ echo
+ echo "ACLOCAL_AMFLAGS = -I ../m4"
+ echo
+ echo "TESTS ="
+ echo "noinst_PROGRAMS ="
+ echo "EXTRA_DIST ="
+ echo "BUILT_SOURCES ="
+ echo "SUFFIXES ="
+ echo "MOSTLYCLEANFILES ="
+ echo "CLEANFILES ="
+ echo "DISTCLEANFILES ="
+ echo "MAINTAINERCLEANFILES ="
+ echo
+ echo "AM_CPPFLAGS = \\"
+ echo " -I. -I\$(srcdir) \\"
+ echo " -I.. -I\$(srcdir)/.. \\"
+ echo " -I../${sourcebase-lib} -I\$(srcdir)/../${sourcebase-lib}"
+ echo
+ echo "LDADD = ../${sourcebase-lib}/${libname}.${libext}"
+ echo
+ for module in $modules; do
+ func_verify_tests_module
+ if test -n "$module"; then
+ func_get_automake_snippet "$module" > amsnippet.tmp
+ # Skip the contents if its entirely empty.
+ if grep '[^ ]' amsnippet.tmp > /dev/null ; then
+ echo "## begin gnulib module $module"
+ echo
+ cat amsnippet.tmp
+ echo "## end gnulib module $module"
+ echo
+ fi
+ rm -f amsnippet.tmp
+ fi
+ done
+ echo "# Clean up after Solaris cc."
+ echo "clean-local:"
+ echo " rm -rf SunWS_cache"
+ echo
+ echo "# Makefile.am ends here"
+ }
+
# func_import modules
# Uses also the variables
# - destdir target directory
***************
*** 811,816 ****
--- 954,960 ----
) > "$testdir/m4/Makefile.am"
subdirs="lib m4"
+ subdirs_with_configure_ac=""
if test -f "$testdir"/m4/gettext.m4; then
# Avoid stupid error message from automake:
***************
*** 821,826 ****
--- 965,1022 ----
subdirs="$subdirs po"
fi
+ if test -n "$inctests"; then
+ test -d "$testdir/tests" || mkdir "$testdir/tests"
+ # Create tests/Makefile.am.
+ sourcebase=lib
+ func_emit_tests_Makefile_am > "$testdir/tests/Makefile.am"
+ # Create tests/configure.ac.
+ (echo "# Process this file with autoconf to produce a configure script."
+ echo "AC_INIT([dummy], [0])"
+ echo "AC_CONFIG_AUX_DIR([../$auxdir])"
+ echo "AM_INIT_AUTOMAKE"
+ echo
+ echo "AM_CONFIG_HEADER([config.h])"
+ echo
+ echo "AC_PROG_CC"
+ echo "AC_PROG_INSTALL"
+ echo "AC_PROG_MAKE_SET"
+ echo "AC_PROG_RANLIB"
+ echo
+ if grep AC_GNU_SOURCE "$testdir"/m4/*.m4 > /dev/null; then
+ echo "AC_GNU_SOURCE"
+ echo
+ fi
+ if grep gl_USE_SYSTEM_EXTENSIONS "$testdir"/m4/*.m4 > /dev/null; then
+ echo "gl_USE_SYSTEM_EXTENSIONS"
+ echo
+ fi
+ # We don't have explicit ordering constraints between the various
+ # autoconf snippets. It's cleanest to put those of the library before
+ # those of the tests.
+ for module in $modules; do
+ func_verify_nontests_module
+ if test -n "$module"; then
+ func_get_autoconf_snippet "$module"
+ fi
+ done
+ for module in $modules; do
+ func_verify_tests_module
+ if test -n "$module"; then
+ func_get_autoconf_snippet "$module"
+ fi
+ done
+ echo
+ # Usually tests/config.h will be a superset of config.h. Verify this by
+ # "merging" config.h into tests/config.h; look out for gcc warnings.
+ echo "AH_TOP([#include \"../config.h\"])"
+ echo
+ echo "AC_OUTPUT([Makefile])"
+ ) > "$testdir/tests/configure.ac"
+ subdirs="$subdirs tests"
+ subdirs_with_configure_ac="$subdirs_with_configure_ac tests"
+ fi
+
# Create Makefile.am.
(echo "## Process this file with automake to produce Makefile.in."
echo
***************
*** 855,869 ****
echo
fi
for module in $modules; do
! func_verify_module
if test -n "$module"; then
func_get_autoconf_snippet "$module"
fi
done
echo
makefiles="Makefile"
for d in $subdirs; do
! makefiles="$makefiles $d/Makefile"
done
echo "AC_OUTPUT([$makefiles])"
) > "$testdir/configure.ac"
--- 1051,1073 ----
echo
fi
for module in $modules; do
! func_verify_nontests_module
if test -n "$module"; then
func_get_autoconf_snippet "$module"
fi
done
echo
+ if test -n "$subdirs_with_configure_ac"; then
+ echo "AC_CONFIG_SUBDIRS(["`echo $subdirs_with_configure_ac`"])"
+ fi
makefiles="Makefile"
for d in $subdirs; do
! # For subdirs that have a configure.ac by their own, it's the subdir's
! # configure.ac which creates the subdir's Makefile.am, not this one.
! case " $subdirs_with_configure_ac " in
! *" $d "*) ;;
! *) makefiles="$makefiles $d/Makefile" ;;
! esac
done
echo "AC_OUTPUT([$makefiles])"
) > "$testdir/configure.ac"
***************
*** 1177,1182 ****
--- 1381,1396 ----
done
;;
+ extract-tests-module )
+ for module
+ do
+ func_verify_module
+ if test -n "$module"; then
+ func_get_tests_module "$module"
+ fi
+ done
+ ;;
+
* )
func_fatal_error "unknown operation mode --$mode" ;;
esac
*** MODULES.html.sh.bak 2005-08-17 16:05:58.000000000 +0200
--- MODULES.html.sh 2005-08-25 01:11:41.000000000 +0200
***************
*** 2098,2106 ****
func_echo '<LI>One or more implementation files: lib/<VAR>module</VAR>.c et
al.'
func_echo '<LI>One or more autoconf macro files: m4/<VAR>module</VAR>.m4 et
al.'
func_echo '<LI>A configure.ac fragment, Makefile.am fragment, dependency
list: modules/<VAR>module</VAR>'
func_echo '<LI>Some documentation'
func_echo '<LI>A POT file and some PO files'
- func_echo '<LI>A testsuite'
func_end UL
func_echo '<HR>'
--- 2098,2106 ----
func_echo '<LI>One or more implementation files: lib/<VAR>module</VAR>.c et
al.'
func_echo '<LI>One or more autoconf macro files: m4/<VAR>module</VAR>.m4 et
al.'
func_echo '<LI>A configure.ac fragment, Makefile.am fragment, dependency
list: modules/<VAR>module</VAR>'
+ func_echo '<LI>A testsuite: source files in tests/ and metainformation (a
configure.ac fragment, Makefile.am fragment, dependency list) in
modules/<VAR>module</VAR>-tests'
func_echo '<LI>Some documentation'
func_echo '<LI>A POT file and some PO files'
func_end UL
func_echo '<HR>'
- gnulib-tool: support for unit test modules,
Bruno Haible <=