Re: megatest fallout

[Ralf Wildenhues]

A bootstrapped megatest directory enables more kinds of tests.
Let's look at these builds:

0) configure CC='gcc -Wall -Werror -fno-builtin'  # as seen earlier
1) configure
2) configure -C
3) pre-seed config.cache with the file resulting from (2), then
   configure -C
4) create a config.cache from only running the ALL test:
   ../../ALL/configure -C
5) pre-seed config.cache with the file resulting from (4), then
   configure -C

6) for each of the sub-configure scripts, do a
   configure -C
   ./config.status --recheck
   and check that neither of the files
     config.status config.cache config.h
   changed between runs.

To compare the results, see the script below.  Ideally, they should
all produce the same results, at least (1)-(6), so any differences are
indication of some bug.  I should note that the order of substitutions
done in config.status for config headers is not stable, so typically,
as soon as there is at least one difference in the file, several more
may appear spuriously.

Actually, most of the differences observed were due to CVS gettext
having a nonzero diff to the respective macro files in gnulib.  It
would be nice to see that reconciled (though I guess I shouldn't
complain, not being in the position to post patches to this end ;-)


Remaining issues (patches in followup messages):

(a) comparing (0) and (1):
error.o in LIBOBJS only in (0); will be fixed in Autoconf's
AC_FUNC_ERROR_AT_LINE.

(b) comparing (1) and (2):
getlogin_r declared in the glob test, but not in the getlogin_r test.

(c) HOST_OPERATING_SYSTEM is set to empty in (2).

(d) While we're at it, the stdout log of these tests also gives a nice
hint as to which tests are worthy of being cached:
  grep \^checking logfile | grep -v cached | sort | uniq -c | sort -k1n

shows
      7 checking whether this system has an arbitrary file name length limit... 
yes
     21 checking for SIZE_MAX... yes

as the top gnulib candidates.  The SIZE_MAX test is cheap on systems
that have it (one AC_EGREP_CPP); the other one has a caching bug, see
followup.

FYI, the Autoconf and Automake tests that show up here:
     39 checking for AIX... no
    351 checking for a BSD-compatible install... /usr/bin/install -c
    351 checking for C compiler default output file name... a.out
    351 checking for style of include used by make... GNU
    351 checking for suffix of executables...
    351 checking whether the C compiler works... yes
    351 checking whether we are cross compiling... no
    352 checking whether build environment is sane... yes

either do employ some sort of (unannounced) caching (install, compiler
tests), or don't do so for good reasons: the C compiler tests are a bit
intricate, and also must obey some backward-compatibility constraints to
older Autoconf versions, the sane-build test may be rather unstable in
time, so is better not cached.  The `make include style' test is quick,
and the old AIX test is not cached in order to discourage its use.

Cheers,
Ralf

#! /bin/bash
set -e

dir1=${1-build2}
dir2=${2-build3}
: ${egrep='grep -E'}

# These lines contain differences (path names, different options;
# ignore LTLIBOBJS because LIBOBJS reveals enough already):
egrep_omit_lines='with options|exec /bin/sh|ac_pwd=|echo..running 
|s,@(ac_ct_CC|CC|CPP|LTLIBOBJS)@,'
# Let's not look at bugs we know about again and again:
sed_ignore_diffs='s| \${LIBOBJDIR}error\$U.o||'

dirs=`ls -1 "$dir1"`
for dir in $dirs
do
  file1=$dir1/$dir/config.status
  file2=$dir2/$dir/config.status
  if test -f "$file1" && test -f "$file2"; then
    output=`diff <( $egrep -v "$egrep_omit_lines" "$file1" | sed 
"$sed_ignore_diffs" ) \
                 <( $egrep -v "$egrep_omit_lines" "$file2" | sed 
"$sed_ignore_diffs" ) || :`
    if test -n "$output"; then
      echo "comparing $file1 and $file2:"
      echo "$output"
    fi
  fi
done