Re: Dynamic loading progress

[Eli Zaretskii]

> From: Stephen Leake <address@hidden>
> Date: Fri, 13 Feb 2015 15:09:08 -0600
> 
> Eli Zaretskii <address@hidden> writes:
> 
> >> From: Stephen Leake <address@hidden>
> >> Date: Thu, 12 Feb 2015 14:09:53 -0600
> >>
> >> I had to add to options to the 'gcc' line in modules/curl/Makefile:
> >>
> >>     -std=c99 -I$(ROOT)/nt/inc
> >
> > Why do you need -std=c99?  What happens if you don't use it?
> 
> It complains about the "register" keyword in string.h stpcpy
> 
> >GCC
> > defaults to -std=gnu99,
> 
> Apparently not in Mingw64

I don't think this is true, as that is a general feature of GCC
independent of the platform.  But in any case, -std=c99 is rarely what
you want, since it's a rare program that conforms to strict ANSI C99
spec.  Use -std=gnu99 instead.

> > I also don't like the -I$(ROOT)/nt/inc thing, it should only be needed
> > for Emacs-specific system-level stuff.  Why was it needed here?
> 
> some file included ms-w32.h:
> 
> address@hidden make
> gcc -std=c99 -ggdb3 -Wall -I../../src -I../../lib `pkg-config libcurl 
> --cflags` -fPIC -c curl.c
> curl.c:1:0: warning: -fPIC ignored for target (all code is position 
> independent)
>  #include <stdio.h>
>  ^
> In file included from ../../src/config.h:1866:0,
>                  from curl.c:6:
> ../../src/conf_post.h:32:27: fatal error: ms-w32.h: No such file or directory

This is wrong on several levels.  First, the module shouldn't use
"-fPIC" unconditionally, as on some platforms, such as Windows, it
invokes a warning and is otherwise ignored.

More importantly, it is dead wrong for modules to include headers in
lib/, and in particular they should NEVER include Emacs's config.h.
That's because a config.h produced for a program or library should
never be used by another program or library -- it includes stuff
specific to that program/library.

On Windows, doing so will get you in trouble because ms-w32.h includes
all kinds of tricks and function redirections/redefinitions that will
most probably break any non-trivial code which interfaces with an
external library.  The above is an example of such trouble, and the
"solution" of add -I../nt -s a wrong solution.

So the question now becomes: what happens if you _remove_ the
"-I../../lib" option from the GCC command line?

If that doesn't work, then we really need an emacs.h file ASAP, and
the module should include only that file, without including <config.h>
inside that file.

> >> Normally those would be added by configure. Perhaps they could be set
> >> via CFLAGS by the perl script when on mingw64?
> >
> > Sorry, I don't understand: the Emacs configure script already adds
> > that directory to the include path.  So what's wrong?
> 
> I'm not running the top level Emacs makefile; I'm running
> modules/curl/Makefile, in order to build the curl module.
> 
> > GNU ld is a one-pass linker, so the libraries should be after the
> > object files.
> 
> Ah, right; I forgot about link order. That fixes the libcurl problem.
> But the original order worked on Debian.

On Debian, you don't need to link against the import library, that's
why.

> >> The other symbols appear to be Emacs symbols? There was some discussion
> >> about a special flag for exporting those on Windows; is that '-Xlinker
> >> -E'? I guess that goes in src/Makefile somewhere?
> >
> > The unresolved externals that are defined by Emacs are expected: you
> > need to link against an import library produced as part of building
> > emacs.exe.
> 
> Right, but I was expecting this makefile to do that. Especially since it
> "just works" on Debian.
> 
> > Otherwise all this stuff will not work on Windows, because
> > the Windows port of the linker must see the import library at link
> > time.
> 
> Apparently that is not true on Debian.

Yes, and I tried to explain why.  Dynamic linking on Posix systems
works differently: the reference to external symbols is left at zero,
and is resolved at run time.  On Windows, these references cannot be
left unresolved, so using the import library allows the linker to
resolve them to an "import symbol" which is just an indirect call
through a dispatch table.

> > The flag to produce the import library should be something like this:
> >
> >    -Wl,--out-implib=libemacs.dll.a
> 
> And that goes in src/Makefile somewhere? Not on the "emacs$(EXEEXT)"
> line, since that doesn't run gcc. So it must go on the temacs$(EXEEXT)
> line (line 693)?

Yes, it should be part of the temacs link command.

> And until we write emacs.h, it also needs -Xlinker -E (or can that
> be -Wl,-E ?).

I don't see what does a header file have to do with linking.  Why do
you think you need that option?

> That should be added by configure when Windows and --with-ltdl are
> specified. 

I'd imagine --with-modules, not --with-ltdl, but I didn't take a look
at the branch yet.

> Hmm. We already have:
> 
> LIBLTDL = -lltdl -Wl,--export-dynamic
> 
> which is included in LIBES, and thus in the temacs.exe link. However,
> that produces a warning:
> 
> C:/msys64/mingw64/lib/gcc/x86_64-w64-mingw32/4.9.2/../../../../x86_64-w64-mingw32/bin/ld.exe:
> warning: --export-dynamic is not supported for PE+ targets, did you mean
> --export-all-symbols?
> 
> So apparently that should be changed to --export-all-symbols? Doing that
> produces libemacs.dll.a

No, using --export-all-symbols is wrong.  See my other message in this
thread.  We don't want to export all the Emacs symbols to the outside
world.  Please use -Wl,--out-implib= instead.

> > Then you need to link the shared library against -lemacs.
> >
> > And the link command line should produce curl.dll, not curl.so.
> 
> Ok, that produced curl.dll. But doing:
> 
> src/emacs.exe -L modules/curl &
> (require 'curl)
> 
> fails, with "eval: Cannot open load file: No such file or directory,
> curl".

That most probably means the module support stuff doesn't tell Emacs
where to find the library.  Or maybe it tries to load curl.so or
something.  I really don't know, you should debug this.

> That same test works on Debian.

GNU/Linux is what Aurélien tested this on, so it shouldn't surprise
you it works on GNU/Linux, but not on Windows.  IOW, it's not an
interesting data point.

> So one question is how to make this Makefile do the right thing
> for the current operating system. The intent is for this Makefile to be
> part of the Emacs test suite, so it needs to not require user intervention.

Indeed, this is part of the job of figuring out how to use modules in
Emacs.  Apparently, that part was not yet done.

> One option is to rename modules/curl/Makefile to
> modules/curl/Makefile.in and use configure in the modules directories.

Which configure? the Emacs configure?  That'd be a wrong solution,
IMO, as it will require the Emacs maintainers to know about modules in
order to DTRT for them in our mainline configure script.

So if Autoconf should be used to generate Makefile's in the modules,
it should be their own configure scripts.

> Another is to set flags in modules/tests.py.

That requires Python, which is an additional requirement for the build
environment.  I'd rather we used Emacs for that.

Alternatively, if the number of different settings is small, we could
assume GNU Make and use its conditional directives to cater to the few
different platforms.  That's much easier, but doesn't scale well to
large and complex modules.  But perhaps that's not a big deal, if we
assume modules will not be large and complex.