Re: recursive configure and some $ac_highest_top_srcdir?

[Steffen Dettmer]

Hi,

thank you for your great detailed reply!

On Wed, Oct 21, 2009 at 10:08 PM, Ralf Wildenhues wrote:
> * Steffen Dettmer wrote on Wed, Oct 21, 2009 at 11:26:22AM CEST:
>> when using recursive configure via AC_CONFIG_SUBDIRS, how to know
>> (in a sub-configure) where the top directory is?
>
> Good question.  If your project structure has a variable level of upper
> configure scripts then, as you describe, you cannot hard-code this.
>
> However, if we follow the ideas behind configure recursion, you also
> should not expect that above the top-most level configure script is one
> that comes from your project: one goal of the uniform argument scheme
> of configure scripts is that I could take your project, plus several
> others, put them side by side in subdirectories, and write an upper
> level configure.ac:
>  AC_INIT([super-package])
>  AC_CONFIG_SUBDIRS([your-package other-package ...])
>  AC_OUTPUT
>
> and your package should cope with that.

Yes, I think this wouldn't be a problem, as long as the packages
needed by "your-package" are available too (and/or specified by
--with-neededpgk).

> This explains why $ac_highest_top_srcdir should not come from Autoconf.

Ahh yes, the packages needed by "your-package" in this case would
not be "super-package/your-package/package1" but
"super-package/package1" and --with-package1=../package1 would
work. But whoever creates super-package would need to know and do
this, right.

> For absolute names, you can easily solve this issue in your own package
> by adding something like
>  if test "${highest_abs_top_srcdir+set}" != set; then
>    highest_abs_top_srcdir=`cd "$srcdir" && pwd`
>    export highest_abs_top_srcdir
>  fi
>
> to each configure.ac that could potentially serve as the highest one in
> your package tree.  (Note you still have to propagate the value from
> configure to makefiles etc.)

Beside that this is absolute (instead of relative), would this
work? I would expect that ./config.status --recheck, which is
called by (auto-)make won't work then, because
highest_abs_top_srcdir would be set differently that in the first
run?

> Dealing with not-necessarily-absolute names looks like it needs
> Autoconf support.  But even then it seems quite tricky.
>
> It would be easy to have an Autoconf API that lets configure correct a
> variables or a command-line argument by the reverse path
> ($ac_top_builddir_sub) upon configure recursion in _AC_OUTPUT_SUBDIRS.

I'm sorry, I don't understand this.
It is easy to correct a command-line argument automatically (i.e.
by prepending a number of "../" prefixes)?

> But would this be elegant, too?  Ideally, it should be possible to
> transport a
>  --with-foo-package=../foo
>
> through a couple of sub configure scripts that each don't care about the
> --with-foo-package switch at all, thus won't have any annotation for it
> either.  OTOH, we certainly cannot correct all --with-* switches that
> way; while it would have been cool to specify such semantics from the
> beginning, it's far too late to do so now.

It would be sufficient to have one single option transformed in
this way. Our stuff around --with-package uses the parameter
../foo as search directory. This code instead could do to use
$transformed_directory/../foo instead.

So with --with-parent (defaulting to ".") beeing transformed to
--with-parent=.. and --with-parent=../.. or alike would be
sufficient.

Is there a way for a toplevel configure to pass a different
option automatically to a sub-configure? Internally it is,
because by some way --cache-file works, could we create something
similar?

This would solve our problem: we would initiate in our highest
level configure (regardless if called by an even higher one or
not, it would work) and all subconfigures would get this option
(and ignore it if not "our" package) but evaluate it in our code
around --with-package.

It would be great if you have any pointers where I can learn
more, please!

> Maybe you can live with something like this: pass absolute directories
> to the --with-* flags.  In the macros that deal with their values, try
> to relativize the directories.  There exist a few ways to do that; they
> typically have problems with symlinks though, and might need adjustment
> for w32-like drive specs.  The one I know is in gnulib-tool, but I think
> I saw a perl version somewhere too.

We had several attempts to map paths from relative to absolute
and vice versa and to "canonify" (cut dir/../dir to dir and
remove "././." etc). By the time, we had to replace all of them
by relative paths because we were unable to get it working for
all cases where we need it. On linux, of course everything works
all the time, but cygwin, MSYS and WINE are troublesome. Usually
it worked only for two of the three...

>> Is there any way to pass some "--topsrcdir=../$topsrcdir" to the
>> subconfigures?  topdir would start with "." if not set and
>> extended in each level.
>
> Not currently, but as I said, something like that could easily be added.

Do you have some key words I could search for to learn more
about this?

>> Of course it would be more complicated
>> for things like AC_CONFIG_SUBDIRS([two/dir_levels]).
>
> Not really.  If you look at the code generated for a AC_CONFIG_SUBDIRS,
> there is a $ac_top_build_prefix computed on the fly which computes the
> needed string.  It's just that you couldn't get at it currently.

Where do I find this code?

address@hidden:/local/tmp/steffen/systest_exp # grep
ac_top_build_prefix ./configure one/configure two/configure
reveals nothing.

>> I understand that such an $ac_highest_top_srcdir would make it
>> complicated (or even impossible) to run a sub-configure directly.
>
> No, I don't see that.

ok, if there is some --with-parent option, user would just need
to set it correctly, otherwise it would default to . but in this
cause it could be needed to have .. or `../..'. But
--with-parent=.. would solve this. Of course ./config.status
--recheck would need to remember the --with-parent passed by the
top level configure (I know it will remember if passed by the
user, because AFAIK it simply remembers all user-passed
parameters, right?).

>> We could "count double-dot levels" in --srcdir or "detect" the
>> depth by counting dots in --cache_file and prepend it to
>> $ac_top_srcdir, but this sucks and I guess it is likely to cause
>> other trouble.
>
> That won't work if I run this with the toplevel:
>  mkdir build
>  cd build
>  ../configure
>  ^^^
> (one more level than you'd think).

why? --cache-file would be build/config.cache and in a sub
configure, lets say packge1, would configure in build/package1
and have --cache-file=../config.cache - and from package1 srcdir
.. is the top sourcedir, so it would be correct I think?
(But anyway, this seems to be a bad option).

>> Absolute paths in --with-me fail at least when using WINE.
>
> Is that some issue in Wine or in Autoconf, or elsewhere?
> The typical thing to write is
>  case $file in
>  [\\/]* | ?:[\\/]* ) $file is absolute ...;;
>  *) $file is relative ...;;
>  esac

All of them, including elsewhere :)
(in autoconf, AFAIK, no real issue just complexities with
quoting).

Some notes from our bug tracker:
> > WINE must not have a drive letter mapped to a directory above
> > the current workdirectory (otherwise, it uses this for the
> > compiler process as cwd, and then the absolute path is "too
> > long" and does not work).
> > When this happens also the dependencies are wrong because
> > including driver letters breaking Makefile where : is special
> > cygwin needs IC_CANONICAL_PATH because the CL.EXE compiler
> > does not understand /cygdrive/c, resulting in C: which in
> > turn is not understood by Makefile so no depencies work.
> > with tcc it works under cygwin when both relative and
> > absolute pathes are used, because when first the relatives
> > are in command line, they are used for deps which accidently
> > works.
> > with cygwin-gcc under cygwin it works.  An idea was to use
> > relatives both with and without $(top_srcdir) but this fails
> > for armlink -libpath seem not to work.
> > update: can we fix the drive-letter-in-deps by extending depcom_arm
> > ohh, now there is something about that limits.h cannot be
> > found by tcc, check if you have a WINE driver letter shortcut
> > mapping such as ~/.wine/dosdevices/x: -> /tmp which fails
> > when you are working in /tmp!

For WINE a problem is that configure runs under linux and sees
lets say /tmp and finds files there and stuff, but compiler sees
some X:\TMP only.

Another problem is when user has mappings
/ -> Z:
/tmp -> X:
and then does some
$ cd /tmp/myworkdir/build
$ ../configure
than absolute paths would be Z:/tmp/myworkdir but WINE will not
set pwd to Z:/tmp/myworkdir but automagically the shorter
X:/myworkdir.

Compiler requires to get a windows-path but make (dependencies,
include ../mymak.mak etc) does not understand things with driver
letters.

On cygwin, this works a bit more when mapping via cygpath
from /cygdrive/c to c: and vice versa, but at least it is
complicated (for me). Also great is handling c:\Program Files\
due to its lovely blank space :(

To run java tools we ended up with something like
address@hidden:/local/tmp/steffen/.../src # make echo-java
"/usr/lib/java/bin/java" "-DILOG_CONSOLE_COLOR=$ILOG_CONSOLE_COLOR"
-cp "`cygpath -w -m -p
../lib/junit.jar:../lib/rxtxcomm.jar:../classes`"
"-Djava.library.path=../lib"

I tried a similar thing for compiler wrappers, but I though its
complexity exeeds everything :)

>> NB: It already is impossible to AC_CHECK_XXX for things from package
>>     (it would be cool to be able to check if package2 has this
>>     and that header or some FUNC, but this won't work, because
>>     -I$(top_srcdir)/../package2/src needs "make" to be evaluated,
>>     so compiler would try a directory "$(top_srcdir)", which does
>>     not exist, and thus never finds the header of course).
>>     Because of the same problem with compiler wrappers
>>     ("CCLD=$(top_srcdir)/../package2/applinkwrap.sh" etc) we
>>     cannot check if the wrappers do work.
>
> But shouldn't absolute names work as arguments to -I, or in CCLD
> settings?

On linux (without WINE), it works well (as long the build
directory is not moved) yes. In WINE, the absolute path may not
exist or work - with my buggy scripts at least. Probably we do
several things wrong or bad...
I think in theory of course there should be some way to map
everything everywhere with stacks of wrappers, but I fear its
complexity would not be maintainable.

Well, thanks for your reply and help so far!

Have a great weekend,

Steffen