critique of gnulib

[Bruno Haible]

Dear Jonas,

Just discovered this critique of gnulib, that you wrote in 2016:
https://gitlab.com/sortix/sortix/wikis/Gnulib

It deserves a discussion.

> Gnulib is a portability layer but ironically fails to be portable,
> as it requires knowledge of implementation details of every supported
> operating system.

Yes, the word "portable" can mean two things:
  (a) a software can be compiled in a different environment without
      modifications,
  (b) a software can easily be modified for the different environment,
      so that it compiles there.

I share the disappointment with you: when I wanted to run "Portable
Common LOOPS" [1] in GNU clisp in 1992, I had to port it myself; it
wasn't portable in the sense (a) but in the sense (b).

For gnulib, sure, when you look into the files lib/mountlist.c or
lib/get_progname_of.c, you will find that often a new OS requires new
code there.

> The gnulib files included [i]n software releases tends to be rarely
> updated and bug fixes in gnulib can take several years to appear
> in most packages.

In which packages have you observed this? We would have to talk with
the maintainers of these packages, in order to understand the cause
of the problem.

> Some parts of gnulib are unnecessary and cause trouble in the modern
> world as it attempts to fix bugs in irrelevant ancient Unix systems.

This may happen, yes, unfortunately, and should be reported to bug-gnulib
when you encounter such trouble.

> Occasionally gnulib wraps standard library interfaces so code can be
> written with glibc assumptions. For instance, whether malloc(0)
> returns NULL or a unique pointer, which can't be detected at compile
> time, and the code should be rewritten to not make such assumptions
> (wrapping malloc might even make it more difficult for code analysis
> tools to detect bugs).

We are assuming that the people who run the code analysis tools do so
on glibc system, where gnulib does not wrap malloc. Developers that
use other OSes as their main OS where they run analysis tools are not
so much in the focus of the GNU project. We do care about them, but not
as much as about developers on glibc systems.

> Gnulib can occasionally cause more trouble than it solves.

These should be rare. When it occurs, feel free to report it.

> Gnulib may even indirectly have exploit mitigation counter-measures,
> as it prevents the standard library with exploit mitigations from
> being used, potentially making vulnerabilities even worse.

I don't understand this argument. Can you back it with some example,
please?

> Gnulib can be especially bad when cross-compiling, as it assumes the
> very worst about the operating system when it can't perform runtime
> tests, which leads to the maximum amount of gnulib being enabled.
> Depending on how much of gnulib is included, in the worst case, gnulib
> might attempt to replace large parts of the standard library.

All autoconf macros in gnulib have cross-compilation support for
glibc / musl systems and for native Windows, because these are the
systems for which cross-compilation is most frequently attempted.
For other platforms, we don't have made this effort, because on these
platforms programs are usually compiled natively, not cross.
And the effect of "replace large parts of the standard library" is not
that bad. Yes, it bloats the libraries and executables a bit. But that
can be afforded.

> The build system tends to assume that if it can't perform a runtime
> test, then you might have a very specific bug that got fixed in
> OpenBSD 1.4, even though you are not OpenBSD, and that OpenBSD 1.4
> is horribly ancient.

Yes, what you describe here is the Autoconf philosophy - test for a
particular bug, not for a particular operating system -, and Gnulib
follows this philosophy for the vast majority of its Autoconf tests.

It's a lesson that people learned in the 1990ies, but that is still
valid today: As e.g. the i18n code from NetBSD ("citrus") gets pulled
into other operating systems, it's no suitable to just test for NetBSD.
Or when you find many BeOS bugs to be also present in Haiku.

> The correct behavior should be assume the very best about unknown
> operating systems, and only assume a bug if the operating system
> is known to have the bug, and require the user to set the applicable
> environment variables with the true answers if gnulib guesses wrong.

No, the practice that you suggest produces more buggy programs,
because
  1. an unknown operating system often pulls in code or design from
     other operating systems,
  2. even in an OS written from scratch, the same mistakes can happen
     as have happened in existing OSes.

> Overall, you will likely find yourself wishing for gnulib to just go away.

>From your perspective as an OS author, yes, the world would be nicer if
all OSes would implement POSIX to the letter and without bugs, and when
your OS equally implements POSIX to the letter and without bugs, all
applications run fine. But that's not how life is.

Also, the perspective of a package developer is not the same as the
perspective of an OS author.

> Build the port (and don't clean its directory). Look at what object
> files got built. Look for any object files that replace standard library
> functionality. Due to the utility functions and mostly innocuous wrappers
> (malloc(0) for instance), it requires a close look to determine whether
> the object file is undesirable.  If you found something that shouldn't
> be used, look at the corresponding m4 file that does the check (for
> lib/foo.c that's often m4/foo.m4) and understand why it was included.
> The configure output and config.log might be of use.

Yes, that's the recipe to follow when attempting to remove useless workarounds
for a given OS. Should I add text like this to the Gnulib manual?

> freadptr.c, freadseek.c, freadahead.c and SLOW_BUT_NO_HACKS and sometimes
> aborting.

Now that you added stdio-ext functions to your OS [2], the issue is fixed for
good and in the proper way (namely, the source code of these functions reside
in the libc of your OS).

> utimens.c not correctly realizing Sortix does the awesome option.

Feel free to report a bug about the corresponding .m4 macro.

> sprintf calls.

What do you mean? sprintf is a standardized libc function.

> If getgroups isn't available, GETGROUPS_T is assumed to int, not gid_t.

Systems which don't have getgroups() most often don't have 'gid_t',
therefore gnulib will make 'git_t' default to 'int'.

Bruno

[1] 
https://www.cs.cmu.edu/afs/cs/project/ai-repository/ai/lang/lisp/oop/clos/pcl/0.html
[2] https://gitlab.com/sortix/sortix/tree/master/libc/stdio_ext