[Bug-gnulib] using gnulib modules in shared libraries on Windows

[Bruno Haible]

Hi,

Here's a proposal that allows gnulib modules to be used in shared libraries
on Windows.

GNU gettext has been using gnulib modules in its libgettextlib.so for quite
some time. The benefit of the shared libraries is, obviously, that it saves
space in the executables (about 150 KB in each of 15 executables). The patches
here make it possible to do the same thing on Windows with the MSVC compiler
and (when/if libtool supports it) also with the mingw32 tool chain.

The general goal here was to keep the patches minimal and as easy to maintain
as possible. The patches come in two chunks:

1) Make some .h files includable in C++ mode, by use of  extern "C" { ... }.
   Also a special case patch to error.[hc] - varargs don't work in C++ mode.

2) Add a special attribute to every global variable that the module exports.
   (There aren't many of them.)

Here's the ChangeLog entry:

1)
        * backupfile.h: Make this file includable in C++ mode: add extern "C".
        * basename.h: Likewise.
        * copy-file.h: Likewise.
        * findprog.h: Likewise.
        * full-write.h: Likewise.
        * pathname.h: Likewise.
        * progname.h: Likewise.
        * stpcpy.h: Likewise.
        * stpncpy.h: Likewise,
        * strcase.h: Likewise.
        * strstr.h: Likewise.
        * xalloc.h: Likewise.

        * error.h: Use ANSI C "..." declarations when compiling with MSVC, even
        though it doesn't define __STDC__ by default.
        * error.c: Use <stdarg.h> when compiling with MSVC, even though it
        doesn't define __STDC__ by default.

2)
        Support for building DLLs on Windows.
        * argmatch.h (argmatch_die): Add DLL_VARIABLE attribute.
        * backupfile.h (simple_backup_suffix): Likewise.
        * error.h (error_print_progname, error_message_count,
        error_one_per_line): Likewise.
        * getopt.h (optarg, optind, opterr, optopt): Likewise.
        * obstack.h (obstack_alloc_failed_handler, obstack_exit_failure):
        Likewise.
        * progname.h (program_name): Likewise.

        * windowsdll.m4: New file.

The patches for 2) look like this:

-extern argmatch_exit_fn argmatch_die;
+extern DLL_VARIABLE argmatch_exit_fn argmatch_die;

By now you certainly wonder
  - what C++ mode (the equivalent of using GCC option "-x c++") has to do
    with Windows DLLs,
  - why only exported variables and not also exported functions need to be
    declared.
Let me explain.

The official recommended way to export symbols from Windows DLLs is:
   - Prefix all extern declarations with _declspec(dllexport) when compiling
     the DLL itself, with _declspec(dllimport) when compiling code outside
     the DLL.
   - Effect: _declspec(dllexport) tells the linker that the symbol is
     exported from the DLL. The LIB utility, which creates an "import lib"
     .lib file from the DLL, creates for each symbol a pointer variable
     named "_imp__symbol" containing the symbol's address.
     _declspec(dllimport) tells the compiler to use (*_imp__symbol) instead
     of symbol itself. Note that in this case, '&symbol' becomes the value
     of _imp__symbol and is thus not a constant expression any more.

The problem with this approach is the amount of Makefile defines that are
needed: For each DLL one needs a LIB<foobar>_DLL_EXPORTED macro that expands
to _declspec(dllexport) or _declspec(dllimport) conditionally. And obviously
I cannot introduce a token named LIBGETTEXTLIB_DLL_EXPORTED into xalloc.h
and other gnulib modules. OTOH, if I use a macro LIB<module>_DLL_EXPORTED for
each gnulib _module_, it's getting messy as well... In other words, this
approach is fine if the source files of the library are not shared with any
other projects. But it is not workable when some gnulib modules are grouped
ad-hoc into a shared library.

The alternative approach, which I've employed in GNU gettext:
   - Prefix all extern _variable_ declarations with _declspec(dllimport).
     Do nothing for function declarations.
     Create and maintain by hand a .def file listing all the exported symbols
     of the DLL.
   - Effect: The .def file replaces the _declspec(dllexport) attributes.
     Function references still work despite of lacking _declspec(dllimport).
     Variable references inside the DLL are less efficient (go through
     *_imp__symbol instead of directly), but work nevertheless. The compiler
     gives a warning when a variable definition is seen and
     _declspec(dllimport) is in effect for it, but the warning is harmless.
   - Limitation: Doesn't work for C++, because of the name mangling of
     functions.

There is a complication, in either approach: As noted above, when 'symbol'
is a variable located in a shared library, '&symbol' is no longer a constant
expression! But is is not rare to write code like this:

/* Long options.  */
static const struct option long_options[] =
{  
  { "add-location", no_argument, &line_comment, 1 },
  ...
};

This doesn't compile any more ("&line_comment is not a constant expression")
in C mode. On ELF systems, "readelf -r" shows that the executable contains a
reloc that is resolved by the dynamic loader at program startup:

0804e5f4  00004605 R_386_COPY        0804e5f4   line_comment

But on Windows, the dynamic linker doesn't do it for you any more!

The solution is to compile those source files which contain this problem in
C++ mode. C++ allows non-constant expressions as initializers of global
variables.

Thus the need to make some .h files includable from a C++ compilation unit.

I've attempted to compile *all* of GNU gettext in C++ mode. But the needed
modifications (some struct declarations, casts between enum and int, casts
of the return value of xmalloc and xrealloc, avoiding the C++ keywords
'this', 'new', 'operator', ...) go against the goal of _minimal_ patches.
So I better stay with the policy to compile as much as possible in C mode,
and as much as needed in C++ mode.

Comments?

                 Bruno

========================== Sample patch for 1) ========================
*** gettext-4/gettext-tools/lib/basename.h      Wed Nov 13 00:24:28 2002
--- gettext-5/gettext-tools/lib/basename.h      Sun Aug 24 03:52:43 2003
***************
*** 1,5 ****
  /* Pathname hacking.
!    Copyright (C) 2001-2002 Free Software Foundation, Inc.
     Written by Bruno Haible <address@hidden>, 2001.
  
     This program is free software; you can redistribute it and/or modify
--- 1,5 ----
  /* Pathname hacking.
!    Copyright (C) 2001-2003 Free Software Foundation, Inc.
     Written by Bruno Haible <address@hidden>, 2001.
  
     This program is free software; you can redistribute it and/or modify
***************
*** 22,27 ****
--- 22,33 ----
  /* This is where basename() is declared.  */
  #include <string.h>
  
+ 
+ #ifdef __cplusplus
+ extern "C" {
+ #endif
+ 
+ 
  #if !(__GLIBC__ >= 2)
  /* When not using the GNU libc we use the basename implementation we
     provide here.  */
***************
*** 29,32 ****
--- 35,44 ----
  #define basename(Arg) gnu_basename (Arg)
  #endif
  
+ 
+ #ifdef __cplusplus
+ }
+ #endif
+ 
+ 
  #endif /* _BASENAME_H */
========================= Sample patch for 2) ===============================
*** gettext-4/gettext-tools/lib/getopt.h        Wed Sep 25 16:19:24 2002
--- gettext-5/gettext-tools/lib/getopt.h        Sun Aug 24 01:44:50 2003
***************
*** 1,5 ****
  /* Declarations for getopt.
!    Copyright (C) 1989-1994, 1996-1999, 2001 Free Software Foundation, Inc.
     This file is part of the GNU C Library.
  
     The GNU C Library is free software; you can redistribute it and/or
--- 1,5 ----
  /* Declarations for getopt.
!    Copyright (C) 1989-1994, 1996-1999, 2001, 2003 Free Software Foundation, 
Inc.
     This file is part of the GNU C Library.
  
     The GNU C Library is free software; you can redistribute it and/or
***************
*** 44,50 ****
     Also, when `ordering' is RETURN_IN_ORDER,
     each non-option ARGV-element is returned here.  */
  
! extern char *optarg;
  
  /* Index in ARGV of the next element to be scanned.
     This is used for communication to and from the caller
--- 44,50 ----
     Also, when `ordering' is RETURN_IN_ORDER,
     each non-option ARGV-element is returned here.  */
  
! extern DLL_VARIABLE char *optarg;
  
  /* Index in ARGV of the next element to be scanned.
     This is used for communication to and from the caller
***************
*** 58,73 ****
     Otherwise, `optind' communicates from one call to the next
     how much of ARGV has been scanned so far.  */
  
! extern int optind;
  
  /* Callers store zero here to inhibit the error message `getopt' prints
     for unrecognized options.  */
  
! extern int opterr;
  
  /* Set to an option character which was unrecognized.  */
  
! extern int optopt;
  
  #ifndef __need_getopt
  /* Describe the long-named options requested by the application.
--- 58,73 ----
     Otherwise, `optind' communicates from one call to the next
     how much of ARGV has been scanned so far.  */
  
! extern DLL_VARIABLE int optind;
  
  /* Callers store zero here to inhibit the error message `getopt' prints
     for unrecognized options.  */
  
! extern DLL_VARIABLE int opterr;
  
  /* Set to an option character which was unrecognized.  */
  
! extern DLL_VARIABLE int optopt;
  
  #ifndef __need_getopt
  /* Describe the long-named options requested by the application.
========================== windowsdll.m4 =================================
contains essentially
AH_VERBATIM([
/* On Windows, variables that may be in a DLL must be marked specially.  */
#if defined _MSC_VER && defined _DLL
# define DLL_VARIABLE __declspec (dllimport)
#else
# define DLL_VARIABLE
#endif
])