Autobuild of gnulib

[Simon Josefsson]

I have started daily builds of gnulib modules on a machine.  The
results so far (the build takes _hours_ to finish) are available from:

http://autobuild.josefsson.org/gnulib/gnulib.html

I know the output isn't easy to parse yet -- I will modify autobuild
to output a summary with the latest build for each project in the same
output file.  Autobuild simply hasn't been used for so many projects
in the same output file yet.  Note that until each module has a self
tests, the "result" column will only print "Built" and not "Success"
for that column.  To have "Success" be printed, the self test has to
run ok.  See for example:

http://autobuild.josefsson.org/gnulib/gnulib.html#arcfour

For an example where there is a self-test but it fails (which results
in a status of 'Almost'), see:

http://autobuild.josefsson.org/gnulib/gnulib.html#argp

The script that you run on a build robot is simple, see below.
Everyone can run a similar script, that submit build logs to
address@hidden, and the results will end up in the
same URL as above.

I'm still experimenting with the setup, but I intend to move daily
builds for several projects to the same system shortly, and provide
the same free service for other projects.  If someone can help and run
the script on a non-i386 or non-linux machine and submit build logs,
that would make things more interesting.

To be able to figure out the host and build system types correctly,
the following patch to gnulib-tool would be useful.  Ok to install
this?  It affects generating the configure.ac for --create-testdir,
and not the configure.ac snippet in --import.  This patch is not
strictly necessary, the daily-buildscript could output the same
information, but may give more accurate information about build hosts.

--- gnulib-tool 29 Nov 2006 21:49:50 +0100      1.199
+++ gnulib-tool 01 Dec 2006 15:41:04 +0100      
@@ -2494,6 +2494,10 @@
    echo "AC_PROG_INSTALL"
    echo "AC_PROG_MAKE_SET"
    echo
+   echo "# For autobuild."
+   echo "AC_CANONICAL_BUILD"
+   echo "AC_CANONICAL_HOST"
+   echo
    echo "m4_pattern_forbid([^gl_[A-Z]])dnl the gnulib macro namespace"
    echo "m4_pattern_allow([^gl_ES\$])dnl a valid locale name"
    echo "m4_pattern_allow([^gl_LIBOBJS\$])dnl a variable"

Btw, does anyone know why --create-testdir runs ./configure + make
distclean?  I'd rather remove that step, since it constitute a
noticeable part of the total build time.  The code snippet that does
the step is:

  if grep '^BUILT_SOURCES *+=' "$testdir/$sourcebase/Makefile.am" > /dev/null; 
then
    (cd "$testdir"
     ./configure || func_exit 1
       cd "$sourcebase"
       echo 'built_sources: $(BUILT_SOURCES)' >> Makefile
       make built_sources || func_exit 1
       cd ..
     make distclean || func_exit 1
    ) || func_exit 1

In reality, almost all Makefile.am will contain something that
triggers the grep command, e.g.:

BUILT_SOURCES += $(ALLOCA_H)

Where typically $(ALLOCA_H) evaluates to nothing.  But generally, what
exactly is achieved by this step anyway, and do we need it?

/Simon

#!/bin/sh

GNULIB="$1"
if ! test -d "$GNULIB"; then
    echo "Usage: $0 <GNULIBDIR>"
    exit 1
fi
GNULIBTOOL=$GNULIB/gnulib-tool

DATEFORMAT="%Y%m%d-%H%M%S"

# Update gnulib cvs.
cd $GNULIB
CVSDATE=`date "+$DATEFORMAT"`
cvs update

# Get module list.
MODULES=`$GNULIBTOOL --list`

# Iterate over modules.
for module in $MODULES; do
    echo Working on module $module...
    (set -x
        : autobuild project... $module
        : autobuild revision... cvs-$CVSDATE
        : autobuild timestamp... `date "+$DATEFORMAT"`
        : autobuild hostname... `hostname`
        BUILDDIR=$GNULIB/build/$module
        rm -r $BUILDDIR
        mkdir -p $BUILDDIR
        $GNULIBTOOL --dir $BUILDDIR --with-tests --create-testdir $module
        cd $BUILDDIR
        ./configure && make distcheck
        rc=$?
        cd $GNULIB
        echo rc=$?) 2>&1 | mail address@hidden
done