Re: [Gnu-arch-users] programming in the large (Re: On configs and huge source trees)

[Alfred M\. Szmidt]

   Alfred: you impressed me by digging fairly deep (e.g., coming up
   with the unexec bogosity in package-framework) so I have a fairly
   long and detailed reply for you.

Should I be afraid?

   And yes, dear trolls, this does remain Arch relevant.

Since arch uses hackerlab/package-framework, it is indeed
relevant. :-)

   My first full-time job, back before I was fully into free software,
   was writing a configure/build system and applying it to a
   reasonably large system (the Andrew toolkit and the suite of
   applications built on that).

Ah, but I was refering to using autoconf, not writting a
configure/building system.

   There are two phases in autoconf (and this is part of its problem).
   One phase is translation of `configure.in' and the other phase is
   execution of `../configure'.

I disagree, it allows more modularity to the whole suite.  It hides
many details that you simple shouldn't need to worry about too.

   The first phase of autoconf relies on GNU m4 and perl.
   Increasingly, in practice, it relies on automake and libtool.
   While jocularly dismissed in the autoconf documentation, the
   circular dependency between m4 and autoconf is a clear
   bootstrapping bug.

I fail to understand why this is a bootstrap bug, yes, there is a
circular dependency there, but not one that causes any grief.  As
compared to some weirder ones--GNU ada anyone?  You'd also have a
circular dependency in package-framework if it gets used, consider the
shell starting to use it, then you need a shell to configure the
shell, and viola, circularity.  Basiclly, you can't come around
circular dependencies for this type of thing, and I don't consider it
a serious bogisity.  For things that produce architecture dependant
output (compilers) this is far more important though.

   The awk dependencies in package-framework are extremely slight --
   easily done away with.  I don't claim that package-framework is the
   right code-base to start-with, only that it's a good demonstration
   of what is achievable.

I don't think that it is a very good demonstration at all.  To me it
is autoconf done badly.

   One thing package-framework, in combination with a portability
   library like libhackerlab demonstrate (to, let's say, a solid
   working-prototype level) is that applications don't need the
   two-phase hair of auto*.


   It's simpler, at least as effective, and certainly more easily
   maintained to have applications above the bootstrapping level
   depend on just a minimal GNU development environment.

I'm not sure what you mean with a minimal GNU development enviroment,
if you mean make, gcc, ..., then your package-framework won't fix
squat.  Why?  Everything already uses autoconf.

   Add to that the number of packages that make poor or outright
   incorrect use of auto*

I agree!!!  But then, many people make poor and outright incorrect
usage of anything and everything.  :-)

   -- and the size of the documentation and

I fail to see how this is a bad thing.  Documentation is good.

   obscurity of the codebase for auto*

That it is written in perl?  Doesn't bother me at all, it is up to the
maintainers.  Just like it doesn't bother me a whole lot how tla's
codebase looks like.

   -- and you should start to wonder who is benefiting from its
   widespread use and why.

For the same reasons why C is in widespread use.  It does the job, and
it allows people enough flexibility to shoot themselfs in the head.

     Alfred> Infact, I think that normal users should simply use
     Alfred> binary packages.  If you are a developer and wish to hack
     Alfred> on something, it is trivial to configure a program.

   Actually, I strongly agree but with a qualification.

And I agree with your qualification.

   The REQS and OPTS dependency sorting stuff is part of a solution.

Which can be implemented to some extent in auto*, wouldn't even be
hard I think.  One could write a couple macros that do the same thing
that REQS/OPTS (not entierly sure what they do) achive.

   So, you're right, but still package-framework has good stuff to say
   about the topic.

Maybe it does, maybe it doesn't.  I'd have to dig far deeper to make a
conclusive opinion about it.  But say that it has some good stuff to
say about the topic, why not add that stuff to autoconf?

   It should suffice, for that purpose, for the maintainer of the
   little GNOME project to publish an Arch-type `config' file.

That is icky, downloading a couple of hundred megs to compile a 100k
program.  So icky that I would just delete the program and not use it.

   And, no -- I'd also want more standardized and better designed
   install conventions so that I can mix a bunch of packages in one
   tree, give one parameter, and have the `--with' stuff filled in for
   all the sub-packages from that (so to speak -- the literal
   mechanism might be different).

autoconf already suppports that (actually, automake), see binutils. :)

     Alfred> Then there is the major deficency of tla using static
     Alfred> libraries for hackerlab.  Assume that you have a dozen
     Alfred> programs using hackerlab, and you find some security
     Alfred> issue or what not in some function, you will end up
     Alfred> recompiling everything.  Simply out of the question when
     Alfred> you have a few hundred programs.

   I think dynamic libraries are overrated and widely abused but, yes,
   they are also sometimes very valuable.

There are cases where static libraries are a better choice, but for
the example I gave with hackerlab, a shared library would be far more
suitable.  But yeah, I can't disagree that shared libraries are
sometimes abused; but then so is everything.  Internal libraries are
libraries that should be static for example.

   The package-framework demonstration *would* have support for them
   if libtool authors had bothered to float their collected knowledge
   of how they work on various platforms in some form other than their
   source code.  There's a few man-months project there to tease that
   information out into a more useful form (ideally making libtool
   itself more data-driven from that database of wisdom).

I hate libtool, and I don't wisht to talk about it. :-)

   Between 2001 and 2004 I made various attempts to download packages
   in source form to a FreeBSD system and build them.  When packages
   had lots of prereqs, config/build/install bugs in auto*-using
   packages was most often the show-stopper.

I'd blame FreeBSD, and not auto*.  I configure quite alot of packages,
and seldom experience any bugs other than in badly written
configure.in's, and badly written source code.

   The two phase thing and its consequences.  Lots of packages wind up
   with "3rd party macros" that may work on Linux but sure didn't on
   FreeBSD.

I still have problem to see how the two phase thing is a problem.  But
badly written macros is something I often encounter, but then, I also
encounter badly written code.  And I'm sure people would abuse
package-framework in ways that would make your head explode.

   Sorry to have mischaracterized you.

No worries.

   It's definately true that naked benchmarks of the `str_' functions
   can't compete with good native `libc' semi-replacements.  It's
   consistently looked to me like this was never a big deal in tla
   performance and the trade-offs (e.g., code so simple it serves as
   documentation) have, so far, been more worth it than not.

As I said, it might not be significant at all, but it might as well
be.  Someone would have to run a profiler on tla and see.

   But, quick thought experiment: suppose RMS decided tomorrow that it
   was desirable to float a simplifed bootstrapping compiler or
   conduct a particular major code cleanup in a systematic way.
   Suppose we wanted 10% of the effort to go in that direction.  What
   do you think would happen?

Considering that it wouldn't happen, I can't imaging what might
happen.  But since the GCC people are quite nice, I doubt anything
serious would occur.  They would probobly come with reasons why it is
bad, or come with reasons why it is a good idea.

   Details matter.  Note that hackerlib, string functions
   notwithstanding, don't actually (despite all claims to the
   contrary) replace much of libc at all.

I stand corrected then, I'm not all that familiar with hackerlab.

   Out of the bulk of `hackerlib' there are a few 10 string functions
   that people complain about and out of *that* alone people construct
   arguments such as yours.

As I said, I'm not all that familiar with hackerlab.  So maybe I'm
just seeing it in a different light that magnifies the libc bits.