RFC: autom4te

[Akim Demaille]

I've just added to the repository (not to the package) my idea of what
autom4te should be.  Of course it is still experimental, the interface
is still somewhat rude, but the core features are there.

autom4te is not just a wrapper around m4 as autoconf is: it implements
a system for caching the traces.  Why?  Because running m4 is
extremely costly, and a typical package needs (will need) to run it
three times: autoconf, autoheader, and automake (which will run
autoconf to fetch the relevant data instead of trying to get it by
hand).

On my machine it takes about 1mn to run autoconf on the fileutils'
configure.in.  Multiplying this by three is a real penalty.


In order to play with it, you first need to be sure that CVS Automake
is installed with the same $prefix as your Autoconf package: this is
to ensure that we can find Automake::Struct.

Then:

~/src/ace % ./config.status --file autom4te && chmod +x autom4te nostromo 13:28
config.status: creating autom4te

And now you can play with it:

~/src/ace % ./autom4te --verbose autoconf.m4f aclocal.m4 configure.in -o 
configure.new
autom4te: the trace request object is:
$VAR1 = bless( [
                 'traces.0',
                 0,
                 [],
                 [
                   'autoconf.m4f',
                   'aclocal.m4',
                   'configure.in'
                 ],
                 [],
                 {
                   'AH_OUTPUT' => 1,
                   'AC_CONFIG_HEADERS' => 1,
                   'AC_LIBSOURCE' => 1,
                   'AC_DEFINE_TRACE_LITERAL' => 1,
                   'AC_SUBST' => 1,
                   'include' => 1
                 }
               ], 'Request' );

autom4te: running: /usr/bin/m4 --define m4_tmpdir=/tmp/t47298.25467 --define 
m4_warnings= --debug=aflq --error-output=autom4te.cache/traces.0 
--trace=AH_OUTPUT --trace=AC_CONFIG_HEADERS --trace=AC_LIBSOURCE 
--trace=AC_DEFINE_TRACE_LITERAL --trace=AC_SUBST --trace=include 
--reload-state=autoconf.m4f aclocal.m4 configure.in >/tmp/t47298.25467/output
autom4te: formatting traces for `/tmp/t47298.25467/dependencies': m4_include, 
include
autom4te: configure.new depends on autoconf.m4f which is more recent
autom4te: creating configure.new
configure.in: 42: undefined macro: AC_SUBSTS

(This complaint is due to the fact that currently autom4te checks the
#-comments too, and Autoconf's configure.in has AC_SUBSTS in some of
its comments).

Fortunately it's the same as running autoconf.

~/src/ace % diff configure configure.new
~/src/ace %

Now let's trace some macros that had never been traced:

~/src/ace % ./autom4te --verbose autoconf.m4f aclocal.m4 configure.in -t AC_INIT
autom4te: the trace request object is:
$VAR1 = bless( [
                 'traces.0',
                 0,
                 [],
                 [
                   'autoconf.m4f',
                   'aclocal.m4',
                   'configure.in'
                 ],
                 [],
                 {
                   'AC_INIT' => 1,
                   'AH_OUTPUT' => 1,
                   'AC_CONFIG_HEADERS' => 1,
                   'AC_DEFINE_TRACE_LITERAL' => 1,
                   'AC_LIBSOURCE' => 1,
                   'include' => 1,
                   'AC_SUBST' => 1
                 }
               ], 'Request' );

autom4te: running: /usr/bin/m4 --define m4_tmpdir=/tmp/t42813.25492 --define 
m4_warnings= --debug=aflq --error-output=autom4te.cache/traces.0 
--trace=AC_INIT --trace=AH_OUTPUT --trace=AC_CONFIG_HEADERS 
--trace=AC_DEFINE_TRACE_LITERAL --trace=AC_LIBSOURCE --trace=include 
--trace=AC_SUBST --reload-state=autoconf.m4f aclocal.m4 configure.in 
>/tmp/t42813.25492/output
autom4te: formatting traces for `-': AC_INIT
configure.in:3:AC_INIT:GNU Autoconf,2.50a,address@hidden

Let's do it again:

~/src/ace % ./autom4te --verbose autoconf.m4f aclocal.m4 configure.in -t AC_INIT
autom4te: the trace request object is:
$VAR1 = bless( [
                 'traces.0',
                 1,
                 [],
                 [
                   'autoconf.m4f',
                   'aclocal.m4',
                   'configure.in'
                 ],
                 [],
                 {
                   'AC_INIT' => 1,
                   'AH_OUTPUT' => 1,
                   'AC_CONFIG_HEADERS' => 1,
                   'AC_LIBSOURCE' => 1,
                   'AC_DEFINE_TRACE_LITERAL' => 1,
                   'AC_SUBST' => 1,
                   'include' => 1
                 }
               ], 'Request' );

autom4te: formatting traces for `/tmp/t4111.25517/dependencies': m4_include, 
include
autom4te: formatting traces for `-': AC_INIT
configure.in:3:AC_INIT:GNU Autoconf,2.50a,address@hidden

M4 has not been run.  A directory autom4te.cache was created with
caches depending upon the paths and files used:

~/src/ace % ls autom4te.cache                                    nostromo 17:04
requests  traces.0           

If I change the input files, say I forget the aclocal.m4:


autom4te: the trace request object is:       
$VAR1 = bless( [
                 'traces.1',
                 0,
                 [],
                 [
                   'autoconf.m4f',
                   'configure.in'
                 ],
                 [],
                 {
                   'AC_INIT' => 1,
                   'AH_OUTPUT' => 1,
                   'AC_CONFIG_HEADERS' => 1,
                   'AC_LIBSOURCE' => 1,
                   'AC_DEFINE_TRACE_LITERAL' => 1,
                   'AC_SUBST' => 1,
                   'include' => 1
                 }
               ], 'Request' );

autom4te: running: /usr/bin/m4 --define m4_tmpdir=/tmp/t44552.25530 --define 
m4_warnings= --debug=aflq --error-output=autom4te.cache/traces.1 
--trace=AC_INIT --trace=AH_OUTPUT --trace=AC_CONFIG_HEADERS 
--trace=AC_LIBSOURCE --trace=AC_DEFINE_TRACE_LITERAL --trace=AC_SUBST 
--trace=include --reload-state=autoconf.m4f configure.in 
>/tmp/t44552.25530/output
autom4te: formatting traces for `-': AC_INIT
configure.in:3:AC_INIT:GNU Autoconf,2.50a,address@hidden

Another file is created, and the cache index is updated:

~/src/ace % ls autom4te.cache                                    nostromo 17:06
requests  traces.0  traces.1
~/src/ace % cat autom4te.cache/requests                          nostromo 17:06
# This file was created by autom4te.
# It contains the lists of macros which have been traced.
# It can be safely removed.

@request = (
             bless( [
                      'traces.0',
                      1,
                      [],
                      [
                        'autoconf.m4f',
                        'aclocal.m4',
                        'configure.in'
                      ],
                      [],
                      {
                        'AC_INIT' => 1,
                        'AH_OUTPUT' => 1,
                        'AC_CONFIG_HEADERS' => 1,
                        'AC_DEFINE_TRACE_LITERAL' => 1,
                        'AC_LIBSOURCE' => 1,
                        'include' => 1,
                        'AC_SUBST' => 1
                      }
                    ], 'Request' ),
             bless( [
                      'traces.1',
                      1,
                      [],
                      [
                        'autoconf.m4f',
                        'configure.in'
                      ],
                      [],
                      {
                        'AC_INIT' => 1,
                        'AH_OUTPUT' => 1,
                        'AC_CONFIG_HEADERS' => 1,
                        'AC_LIBSOURCE' => 1,
                        'AC_DEFINE_TRACE_LITERAL' => 1,
                        'AC_SUBST' => 1,
                        'include' => 1
                      }
                    ], 'Request' )
           );


Finally, if an input file is updated, m4 is run again:

~/src/ace % ./autom4te --verbose autoconf.m4f aclocal.m4 configure.in -t 
AC_INIT |& grep running
~/src/ace % touch autoconf.m4f                                   nostromo Err 1
~/src/ace % ./autom4te --verbose autoconf.m4f aclocal.m4 configure.in -t 
AC_INIT |& grep running
autom4te: running: /usr/bin/m4 --define m4_tmpdir=/tmp/t4369.25571 --define 
m4_warnings= --debug=aflq --error-output=autom4te.cache/traces.0 
--trace=AC_INIT --trace=AH_OUTPUT --trace=AC_CONFIG_HEADERS 
--trace=AC_DEFINE_TRACE_LITERAL --trace=AC_LIBSOURCE --trace=include 
--trace=AC_SUBST --reload-state=autoconf.m4f aclocal.m4 configure.in 
>/tmp/t4369.25571/output







I'd like to move the whole Autoconf as quickly as possible to
autom4te, i.e., for 2.51.

How does it sound?