Re: gnulib and threaded execution

[Bruno Haible]

Hi Ralf,

> When you try to use gnulib in threaded
> code, any process-global state can potentially cause problems, whether
> that be static data, file descriptor state, current directory, umask,
> etc.  For a lot of these data and in a lot of the cases, gnulib is safe.
> Still, it might make sense to make an effort to document exceptions to
> this rule: users might not even be aware that three levels down the
> module dependency tree, they are using something potentially unsafe.

Very true. When using fstrcmp in a multithreaded situation in msgmerge,
I had to chase the module dependencies and then look at each involved
source code file. It's doable for fstrcmp(), but when you look at
larger pieces of code, the task becomes daunting.

I would find it useful and systematic if we started to add a
section "Multithreading:" to the module descriptions. Its contents
could be a simple statement like "MT-safe", or a description like
this for iconv_open:
  A conversion descriptor can not be used in multiple threads
  simultaneously.
or for localcharset:
  Depends on global state: The current locale. This function is
  unreliable if setlocale() is called in other threads.

Filling in this information requires a review of the code, looking
at global variables, system calls, and more.

> I egrepped for '([     ]static |static [^()]*;)' (TAB inside) and
> among the first few hits (of a few hundred) I found these issues:

This is just a portion of what needs to be looked at. There's also
the global variables (use 'nm' on the object files to find them),
sigaction() system calls, fork()/exec() calls (which operate in
platform dependent manner if threads are active), and many more.

> * Unless STACK_DIRECTION is defined, gnulib/lib/alloca.c sets and uses
> STACK_DIR and find_stack_direction:addr in the first alloca call.  When
> that first call is from threads, and racing with another one, the value
> for STACK_DIRECTION may be computed wrongly, and the code may corrupt
> the stack.

You're just scratching the surface. More importantly, the 'alloca'
module cannot be used _at_all_ in code meant to be used in multiple
threads.
  1. because the stack size for threads is often smaller than
     the stack size of the main thread. (16 KB vs. 8 MB, or so.)
     You have to use module 'safe-alloca' instead.
  2. because the alloca.c code assumes that there is a "stack
     direction". This is not the case any more with GCC's new
     "split stacks" <http://gcc.gnu.org/wiki/SplitStacks>.

> * error_at_line.c has a function-static old_file_name

Additionally, if error and error_at_line are called from
multiple threads, the output will intermingle on stderr.
So, locking is necessary. In error.c it is not enabled in
gnulib.

> * in getloadavg.c, getloadavg_initialized should probably be a
> sig_atomic_t not a bool (no idea whether this can ever be a problem in
> practice[1]).

I'd say, this looks more like one needs to use a gl_once_t or
pthread_once_t, to guarantee that the initialization is done
at most once (and not started in parallel by several threads
if the second thread arrives when the first thread has started
the initialization and is not yet done with it).

> * register_close_hook is not thread-safe.

Yes, this should be part of the MT related documentation:
which functions are meant to be executed only once, before
anything else.

Bruno