Re: My swarm compile for OSX

[Marc-Antoine Parent]

Le Jeudi, 16 janv 2003, à 19:46 Canada/Eastern, Bill Northcott a écrit :

> > Yes, that is another piece of work. Making it a framework would be 
> > nice.
> > Though I must say, much as I love the framework idea for 
> > encapsulation,
> > I hate the way that any code that uses the framework must use the
> > framework name as a prefix in the include directives! Changing the
> > Makefiles to use the -framework flag is easy enough, but changing the
> > source code to adjust to the Apple syntax is a major thorn in the
> > backside. Think that all Swarm programmers will have to add an #if
> > darwin to all their code... What is Apple thinking? Anyway, sorry for
> > the little rant.
>
> There is absolutely no need to change the source code.  Take a look at 
> the
> Aqua Tcl/Tk projects.  In tcl 99% of the code s identical.  Indeed
> tcl8.4.1 uses the Foundation framework on MacOS X even when you do a 
> UNIX
> style build with confgure and make!  Even with tk the only code that is
> different is the stuff to emulate X11 on the Aqua display.  Even 
> better,
> we can potentially package tcl/tk/BLT/hdf/swarm into a single 
> Framework.
> That way you can build Swarm applications in PB by designating a single
> framework for includes and links.  I am sorry, but this is light years
> ahead of the current porridge of includes and links with all their 
> complex
> dependencies.
>
> All this stuff about frameworks being a problem is just FUD from the 
> usual
> bunch of Luddites.  No less a luminary than Linus Torvalds has made it
> clear that he thinks the GNU people should be taking a long hard look 
> at
> frameworks.

OK. I did not intend to get into a frameworks debate, but here I go.
First, as I said, I love the idea of how frameworks encapsulate and 
version all libraries.
Second, I do think we should eventually package tcl/tk/BLT/hdf/swarm 
each in their own little framework, so they each use the latest 
compatible version of one another. Great technology indeed.
But I don't think you read what I said about the code change carefully 
enough. I was not claiming that the Tcl/Tk code was changed much for 
framework bundling; it was not.
I was making the claim about client code.
A piece of code that used to #include "tk.h" now has to #include 
"Tk/tk.h", provided gcc is told nothing more than -framework Tk as it 
should. (Same if you work within project builder.)
I did circumvent this in another porting project by explicitly adding 
the Header directory of the framework to the PB search path. YUK. Or 
change all the include directives in all source files to read:
#if _DARWIN_
#include "Tk/tk.h"
#else
#include "tk.h"
#endif

Actually, I almost did this, and probably should have. I agree my 
current solution in a hack. But, as far as I'm concerned, so is the 
#ifdef one. I chose the least disruptive of two evils... And cringed.
Why? I agree that the Tk/tk.h syntax pushes file namespace problems one 
level further, and that's a good thing. But why make it necessary? 
IMHO, it should be an optional syntax: -framework should amount to a -L 
so you can leave an existing codebase intact, AND allow the 
Framework/File syntax as an option if you want to be precise and avoid 
conflict when there is a possibility of conflict...

Now, am I missing something fundamental? Maybe. I am not an OSX (or 
unix) guru. But this is not my first porting effort either, and I 
cringe whenever I have to deal with frameworks. If frameworks were as 
convenient to use as they are (admittedly) useful, why is not Apple's 
X11 a framework? (To name one...)

> > > Putting stuff in the root directory is not nice
> > > - one of the many reasons that I don't like Fink.
> > It's quite well circumscribed, actually. And close to being standard 
> > on
> > the platform. Do I understand you won't favor a fink install of swarm?
> > ;-)
>
> Like 'no way Jose.'  Fink is NOT well circumscribed.  It patches the 
> path
> to put /sw/bin at the front.  That means it changes the default code 
> for a
> large and not obvious set of utility commands.  From the support point 
> of
> view it is right back to the old UNIX nightmare.  If someone has MacOS 
> X
> 10.2.3 you know exactly what the default BSD utilities are: which ones 
> are
> usable and which ones are not.  If Fink is installed you no longer have
> clue. It is your typical LINUX mess.

Well... I see your point, but the fact is that some of the Darwin 
utilities are problematic/outdated, (sed being a case in point) and 
without fink I see many people build and install those utilities over 
the darwin ones to get the functionality they need, and you have even 
less of a clue of what is there or not! But it's also true that they 
have local/bin to do that, so why /sw... I tell you, I curse when I get 
apps compiled to the prefix /usr/local, as they are even harder to 
disable at will than the fink ones! (as opposed to 
prefix=/usr/local/appName, which is cool.)

I see your point, I hope you see mine. I certainly enjoy having a handy 
package system, tell you the truth. Helps me focus on the apps I care 
about.

> Fink might have been excusable in the days of 10.0 or even 10.1, but
> Jaguar has a very complete set of BSD commands and a large number of 
> GNU
> ones out of the box, and Apple is committed to add more.  We are 
> trying to
> make Swarm easier to use.  Every single non-standard system patch 
> makes it
> harder.
>
> > I tried to compile a few apps with the apple compiler. No go :-(
> > Also, I said in my email "a recent FSF gcc"; we determined with Alex
> > that it _has_ to be the 3.3 stream. Forget 3.2, as you said.
>
> You would need different CFLAGS: and OBJCFLAGS: -no-cpp-precomp
> -no-long-double (instead of -no-long-long).  Have a look at the
> configure.in files in the CVS code.  Your Makefile.common will also be 
> all
> wrong!  Re-running configure specifying the Apple compiler might 
> rebuild
> it.

Makes sense.... Though  -no-cpp-precomp  is only a way to speed things 
up. -no-long-double is worth investigating.

> What errors did you get?

gcc crash.

> > > > > libtools from libtool-1.4.3
>
> > I am not honestly sure this is necesssary, BTW. I added it because you
> > were considering it. But my new make-in-progress seems happy without
> > it. Yes, all is fine with the CVS version. Oh, but it does introduce
> > -lavcall itself! I wonder what changed.
>
> It may not be strictly necessary, but libtool, config.sub and 
> config.guess
> are all installed by the libtoolise utility.  So the developers think 
> they
> need to be consistent.  We had to update the config things.  So we 
> should
> update libtool (ltmain.sh) to match.

Sounds good.

> > > Hopefully this will be in the cvs by the end of the week.
> > > libtool-1.4.3 also needs a SED environment variable which point to a
> > > recent GNU sed.  It seems you must of had one set up.  I will try to
> > > get
> > > this incorporated into the cvs.
>
> > Yup. Fink sed and gperf and libdl...
>
> Exactly my point about Fink.  Next you try compile something that uses 
> BSD
> sed specific tricks and it breaks - yecchhh!

Well, the original sed was not doing well either, was it? I got things 
working with this one.

> > > > y gcc was built with
> > > > ../gcc-3.3/configure --prefix=/swarm --enable-languages=c,objc
> > > > --enable-shared --enable-version-specific-runtime-libs
> > > > Note to Bill : I saw you were wondering about
> > > > --enable-version-specific-runtime-libs: I ensures that the gcc
> libobjc
> > > > is in $prefix/lib/gcc-lib/powerpc-apple-darwin6.3/3.3 instead of
> > > > $prefix/lib, which avoids collisions with the swarm libobjc (I use
> the
> > > > same prefix)
> > By collision, I mean overwrite. If I use the same prefix for gcc and
> > swarm.
>
> Keep the non-standard gcc well away from anything else!

Yes, I agree that was not wise.

> > I understand you try to be apple compatible, but as long as we're not
> > using the apple gcc, we can have safer variants; as for when we do use
> > the apple gcc, it won't matter since we won't use the prefix /usr to
> > build swarm, and no conflict will be possible.
>
> Maybe.

Small issue; few people will compile swarm with the gcc as I did. I did 
that because, well, that gcc has no reason to exist on my machine but 
swarm. Still silly, but others may do the same. Or not. I won't lose 
sleep on this one.

> > > Please see the README.MacOSX and INSTALL.MacOSX files.  The correct
> > > place
> > > for OBJCFLAGS is configure.in and libobjc/configure.in.  Then run
> > > autogen.sh, configure and make.  Or more simply open the PB project
> and
> > > build the targets autogen,config and swarmlibmake.  (Note Swarm is a
> > > composite target that does all three)  The rest of the configure
> > > instruction goes in macosx/swarmx/configure.sh
>
> > OK... It's just that I am not that familiar with configure.in, and
> > avoid touching it when I can. Pardon my ignorance in that domain. I'd
> > be happy if you added the flags in the right places now that I
> > determined that they are needed ;-)
>
> autogen.sh runs autoconf and automake.  These create configure files 
> from
> configure.in files and Makefile.in fles from Makefile.am files.  So a
> change to configure.in potentally changes every Makefle in the project.

That's precisely why I'm usually afraid to touch it!

> > > > After that, all seems to compiles fine until the tool directory; but
> > > > there is a hidden problem.
> > > > I can compile the tool (I know you think this is a hack, but imho it
> > > > promotes understanding of the issues ;-) with the following  
> > > > command:
>
> It is particularly important that we get this to work without hacks.  
> The
> build mechanism in the tools directory seems to be the same as that 
> used
> for building Swarm applications.  So it affects all users, not just the
> library builders.

Yes, I suppose that's true; though once the libobjc is in SWARMHOME it 
gets picked up by the application makefiles just fine.

> > > > put libavcall, as I now have to explicitly link it with everything I
> > > > build! Another story.)
>
> That is what I mean.

Yes, and as I said I agree that the work is not over.
Note that in the version I posted, this problem does not exist, since 
the libtool was reverted by my last reconfigure.

> > > avcall is assembler code, so the problem may be that your FSF gcc is
> > > using
> > > the GNU assembler rather than the Apple one.
>
> > OK. This is really puzzling. The avcall that works is the one Alex
> > built using gcc 3.2.1 (If I'm not mistaken.)
> > Alex, had you built the gnu as, ld tools at the same occasion?
> > My avcall (using gnu 3.3) fails its own tests; should we report this 
> > to
> > avcall, gcc or both, you think?
> > And building avcall with the Apple compiler passes its own tests but
> > does not work with the rest of the swarm code.
> > AFAIK, the same assembler is used throughout, no? I mean unless the 
> > gcc
> > build also builds the assembler at the same time?
>
> If you are using Fink, who knows!

Now, you are being superstitious! No, I do not have as or ld in my 
/sw/bin. There!

> > I am quite sure this is not the case, as (when I was making those
> > tests, i.e. before the avcall problem fixed itself >:-( ...) any link
> > would fail, complaining avcall was missing. So I am quite sure avcall
> > was just absent from everything. Also, putting Alex's library in place
> > of mine would fix swarm executables without a relink.
>
> Remember MacOS X is a system based on runtime linking.  When you run 
> the
> linker during the build, you usually just insert a reference to a 
> library,
> not the code.  When the the app is run dyld links the actual code
> dynamically.  This is so that updates to underlying libraries and
> frameworks propagate to existing code without a recompile, which is 
> cool.
> If dyld cannot find the symbol at runtime, you have a problem.

I know that much. Only true of dynamic libraries, of course.

> > Oupse! (insert later email)
> > I thought again about some of what I said about avcall: I have to take
> > back the last claim. Changing the avcall library without relinking
> > cannot change anything as Alex's library is static!
>
> My case rests!  ld adds the code to the linked libraries solving your 
> run
> time problem.

No, no. avcall never got added to the other swarm libraries. It needs 
to be linked in explicitly. (libavcall.a, a static library)
It is not included, or even referred to, in any of the swarm dynamic 
libraries.

> > But I re-tried to build without avcall, or with various versions of
> > avcall, and I stand by my other claims:
> > Even now, no library but libavcall.la contains ___builtin_avcall in 
> > any
> > shape or form that can be seen by nm;
>
> which is why there is otool.

I was being rhetorical. Rephrase:
Even now, no library but libavcall.la contains ___builtin_avcall in any
shape or form.
Unless you are saying that otool sees symbols nm does not? It sees 
other things, but not more things, AFAIK.
Or tell me how to do so, and I will reascertain my previous claim with 
otool.

> > Trying to link any swarm executable unless libavcall is available;
> > building avcall using the CVS-fresh FSF gcc 3.3 creates a corrupt
> > avcall that does not pass its own tests;
> > Linking with an avcall built using the Apple gcc (december) allows
> > self-test success but makes crashing swarm executables;
> > Linking with Alex' avcall (made statically with gcc3.2.1) works.
> > Now, I wonder about one thing: Could the keyword here be statically?
> > No, I just tried compiling libavcall statically (w/o -fPIC) (in gcc
> > 3.3) and it still fails self-tests. Oh, well...
>
> -fPIC has no effect on Darwin/MacOS X.  All code is position 
> independent.
> The only flag needed to compile code for shared libraries is 
> -fno-common.
> I recommend the Apple UNIX porting guide, or the recent O'Reilly book:
> MacOS X for UNIX Geeks.

I read through the former. Not the latter. I might.
I knew about -fPIC, but I was being superstitious myself about why 
avcall fails for me and not for Alex.

> > For me, the most urgent is adding my three sourcecode fixes to the 
> > CVS:
> > a) The dispatchless drop code in DefObject.m
>
> I have asked the others to test that on x86.  If it does not cause
> problems, we will add it.  Changes to C or ObjC source code have to be
> tried on other platforms.  Given the amount of testing that Swarm has 
> had,
> we need very strong reasons to change core code.

Cool.

> > b) The blt library rename in src/tkobjc/TkExtra.m (we should ifdef 
> > that
> > with a BLT version number I suppose. I wish I knew when that
> > happened...)
>
> What is that problem?  I don't have it, but then I don't use Fink.<G>  
> On
> second thoughts, I don't specify the location of BLT only tcl.  That 
> finds
> tk and BLT automatcally

I was referring to contents in my previous email. Should have been more 
explicit again.

Look in blt2.4z/library, you'll find a file called graph.tcl
It gets put somewhere or other in the install, in my case /sw/lib/blt 
since I use fink.
That file was named bltGraph.tcl in earlier versions of BLT (when? I do 
not know BLT)
and Swarm looks for it under that name. (Two lines in 
src/tkobjc/TkExtra.m:

45:   const char *fileName = "bltGraph.tcl";
117:   [self eval: "source $blt_library/bltGraph.tcl"];
)
So those lines should be modified so they look for the file under its 
new name, possibly #if the BLT version > some unknown number.


> > c) Prevent the duplication of nil_method between libobjc/nil_method.c
> > and src/defobj/internal.m
>
> What problem is this the solution for?

The linker complains about duplication:

/usr/bin/ld: warning suggest use of -bind_at_load, as lazy binding may 
result in errors or different symbols being used
symbol _nil_method used from dynamic library 
/usr/local/lib/swarm/libdefobj.dylib(internal.lo) not from earlier 
dynamic library /usr/local/lib/swarm/libobjc.0.dylib(nil_method.lo)


> Thanks for all your thoughts and hard work
> Bill

Sure, I guess.

Marc-Antoine


                 ==================================
  Swarm-Support is for discussion of the technical details of the day
  to day usage of Swarm.  For list administration needs (esp.
  [un]subscribing), please send a message to <address@hidden>
  with "help" in the body of the message.