RE: MSYS+MSVC for libtool branch-2-0, take 7

[Peter Ekberg]

Ralf Wildenhues wrote:
> * Peter Ekberg wrote on Tue, Aug 09, 2005 at 10:18:03AM CEST:
> > Ralf Wildenhues wrote:
> > > Thanks a lot for the overview.
> > 
> > No problem.
> > 
> > > I have a couple of comments for stuff you have not split out yet,
> > > see below (and a couple of issues I have not looked at yet; but I
> > > did not want to wait even longer with this mail).
> > 
> > Regarding splitting things out, How small chunks should I make?
> 
> Ideally: small enough so they are still self-contained, plus easy to
> review.  Realistically, time is finite, so don't waste too 
> much on this.

Great.

> > I mean, the "lib as archiver" seems like a unit. As is the case
> > with "dumpbin as name lister". The rest is also a unit in some
> > sence as it's all to do with support for MSVC, perhaps a natural
> > boundary is C vs C++. Should I also split C support into basic
> > support and fixes for various elaborate test cases?
> 
> Naa, do three patches: "lib as archiver", "dumpbin as name lister",
> and MSVC support.  Maybe split out the file_magic_glob part too as
> it's easy to review.

Ok.

> I'm still very unsure about the "lib as archiver" patch, esp as to the
> naming of the new variables.  Maybe
>   $AR $AR_FLAGS $AR_OFLAG$libname $objects
> is better, with $AR_OFLAG empty on *nix, and $AR_FLAGS empty 
> for "lib"?
> Maybe such a style is easier to copy for Automake?
> (Better keep this discussion to the respective thread, though.)
>
> > Do you have any guidance on how I generate such a patch
> > series using cvs, and how I keep it up to date? Should I just
> > split it out bit by bit and do it the slow way, one patch at
> > a time? I could manage when it was a handfull of unrelated
> > oneliners, but this is more intricate...
> 
> I've always wanted to look at quilt..
> but what I actually do is just a bunch of diff output that 
> gets applied
> and reverse applied.  If you want to know the ugly way: When splitting
> up a large diff, I just go through it linearly and move the chunks as 
> I go along.  No, this is far from optimal or "nice".  Once you have it
> separated, things should work more or less straightforward.
> 
> I believe your patch and Keith's proposed patch for "-sobase" interact
> (i.e., -sobase support would have to be forward ported into 
> your patch.)

Seems like a simple s/libname/sobase_name/ but someone has to
remember to do it...

> > > >   cl -Tp foo.cc -Tp bar.cc
> *snip*
> > > Hmm, I thought there were a non-position dependent option for 
> > > cl.exe to
> 
> > > so you should be fine with CC='cl -TC'.  Does that work with your
> > > version as well?
> > 
> > (Sorry for not mentioning -TC and -TP, but I ruled them out as
> >  useless, maybe that was hasty, but I can't think of a way to
> >  use them)
> > 
> > I have them, but CC='cl -TP' is no good. The -TP switch has the
> > bad effect of really compiling *all* files as .cpp, which is
> > not what you want. If you for example do
> >     cl -TP foo.cc wsock32.lib
> > the dang tools will barf on the library not being C++.
> 
> Ouch.
> 
> > So, the only way to use -TP and -TC is to always compile first
> > (with -c switch) and then link. I think libtool always does it
> > this way so that may not be a problem?
> 
> Yes, libtool requires separated modes for compilation and linking.

Good, good...

> > But you still can't add -TP to $CC as $CC is also used to link.
> > So you have to add -TP to CPPFLAGS, but I didn't find a
> > corresponding variable for C++, so you want different CPPFLAGS
> > for C and C++. Is there such a beast as CXXCPPFLAGS or whatever
> > it may be called? Can tags be of use here? (I'm pretty ignorant
> > when it comes to tag variables)
> 
> We could just weed out -TP and -TC in func_mode_link.  Then 
> you can put
> them in CFLAGS/CXXFLAGS, or even CC/CXX, which I believe is a good
> thing.  CPPFLAGS would have been more correct, but it's 
> shared between C
> and C++.

This is possible yes, I'll look into it. I need to update the patch
anyway, now that all the small stuff have been commited.

> > > Still, we should think of changing ac_ext so it works out 
> of the box.
> > > Doesn't this need a patch against Autoconf?
> > 
> > One reason to change ac_ext to .cpp is that the sources in
> > the tests use .cpp, so that is what you should test for.
> > Perhaps not a great reason, and I'm biased as I'm a lazy
> > person...
> 
> Well, the question is not how to make it easier for the testsuite to
> work but for the users in the end.  They are not going to rename their
> files too *.cpp, so we'll have to cope anyway.

Yes...

> > > > Also, since MSVC uses @ and ? in C++ symbols (it also
> > > > uses @ for __fastcall C symbols for that matter) I'm
> > > > unsure of how ltdlopen (and preopen) will work...
> > > 
> > > Later.
> 
> Let's just see about this when we something breaks.
> 
> > > > * Support for NM="dumpbin -symbols" as the name lister.
> > > 
> > > Nice!
> > 
> > Maybe not so great though. I recently downloaded the free
> > C++ 2003 Toolkit and the latest Platform SDK and dumpbin
> > was not included. I wonder if support for this tool has
> > been dropped? Does anybody know?
> > I did a search on MSDN and there is no information
> > provided in the Visual Studio 2003 docs, when I look
> > in the place dumpbin was documented in previous MSVCs.
> > Dropping support for dumpbin is not mentioned in changes
> > so perhaps the docs has just moved. I don't think so
> > however, it looks like dumpbin is a goner.
> > 
> > And I can't find a tool that can fill the gap. Crap! Help!
> 
> Hmm.  Online MSDN docs don't seem to reveal any deprecation on first
> sight.

No, but it hardly good marketing to have one less check mark
on some feature list, so I'm not all that surprised if it's
been silently nuked.

Hmmm, I just checked the preliminary docs on MSVC 2005 and
dumpbin is there, I wonder how I missed that last time?

http://msdn2.microsoft.com/library/

Decend into "Visual Studio 2005 Documentation" ->
"Visual Studio" -> "Visual C++" -> "Building a C/C++ Program" ->
"C/C++ Building Reference" -> "C/C++ Build Tools" ->
"DUMPBIN Reference"

I wonder why dumpbin is not in the C++ 2003 Toolkit...

> > Maybe I should just give up and say that some tools from
> > binutils are required. No big deal really.
> 
> Well, if we can make the code work in both cases, even better.
> If not, OK by me to just use `nm' then -- it's very likely to
> be present due to mingw anyway.
> 
> > The "lib as archiver" patch is still needed though, as
> > this seems like the easiest way to get at all symbols
> > from a library with so many objects that the max command
> > line length is exceeded (pdemo.test), as the Microsoft
> > linker does not seem to support reloadable objects.
> > You *could* extract symbols from one object at a time
> > and merge the outcome, but I think that would be a much
> > bigger change.
> 
> I'm still not convinced that we don't end up having more trouble with
> paths encoded into the archive.  But since "lib" does that anyway, I
> guess we might just have to care for all bad cases (see that we don't
> encode paths, but expect encoded paths in other libs).  
> 
> I must admit I haven't checked your patch closely w.r.t this yet.

In the patch, the archive built to get the symbols is not used
for anything but getting the symbols. But two features in lib
are used that are not supported by ar. Namely that lib can take
a command file with inputs (@cmdfile notation) so that the
command line length is short, and the fact that the path is indeed
stored in the archive, so that object file renaming is not needed
for this temporary symbol extracting archive.

So, for this, it is not really needed to have AR=lib. It is perhaps
cleaner to look up "lib" (or "link -lib") and use that without looking
at $AR at all, since ar does not have the required functionality.

When doing the actual linking, a command file is again used,
so it's similar to the linker script approach taken with gnu ld.
(except that the linker script isn't used with gnu ld when the
command line length is overrun in the extract symbol step, in
that case the reloadable object file is used for both symbol
extraction and actual link)

> > > > Here's the output from make check, when configured with:
> > > > configure CC=cl CFLAGS=-MD CXX=cl CXXFLAGS=-MD LD=link 
> > > AR=lib STRIP=:
> > > > NM="dumpbin -symbols" F77=:
> > > > 
> > > > This is my environment:
> > > 
> > > Could you be bothered to run the new test suite as well for this
> > > environment?
> > >   make check TESTS=
> > > (adding TESTSUITE_FLAGS="-v" or also "-d" comes in handy 
> at times, but
> > > the autotest test suite saves a lot of interesting state 
> by default.)
> > > 
> > > Show testsuite.log and whatever else you deem interesting.
> > 
> > You mean "new test suite" as in "test suite in HEAD", right?
> 
> Yes.  Could you try it?

In a bit, I have to move the patch to HEAD first...

Cheers,
Peter