MacOS X is not Linux!

[Bill Northcott]

I feel the need for a few words about why MacOS X and particularly its 
code library system is not Linux.  It has taken me a lot of time to get my 
head around this.  So I suspect there are others in the same state of fog 
as I was.

Most of the current Swarm users are on Linux or Cygwin.  This is hardly 
surprising because these are best supported platforms and binaries are 
available through the good work of Paul and others.  Cygwin being a GNU 
guest OS layer on Windows is still very Linux like.

Inevitably people are coming to MacOS X and trying to use it like Linux. 
It is not Linux.  It is a species of UNIX like Tru64, HP-UX, AIX Solaris 
or what ever.  It worth being mindful of the story that GNU is a recursive 
acronym for' GNU is NOT Unix'.

There are two major areas of difference:
1. Format of excutable files - Linux and Solaris use ELF, MacOS X uses 
Mach-O, Tru64 uses ECOFF (if my memory serves) etc.
2. The way dynamic libraries are implemented.

The first problem is easily fixed by using the system's version of ld.

The second difference has much wider implications.  In particular for 
MacOS X:
1.  static linking is strongly discouraged.  So all the build tools, gcc 
(even built from FSF code), ld , libtool etc, build shared library code by 
default.  gcc builds position independent code.  (-fPIC is redundant.) 
Indeed I am not sure that it is possible to not build position independent 
code.  Why would you want to? ( -fno-common must be used because dyld does 
not allow common symbols.)
2.  The build time linking system is very different.  To build a static 
library, one needs to use the -static flag, otherwise it will be dynamic. 
If, when linking one library as dynamic,  you use symbols from another 
dynamic library, the symbols' code will NOT be inserted in your new 
library.  All that goes in is a reference to the name of the library 
containing the symbol's code.  This is what is likely to happen, when you 
build Swarm, because unless you tried hard any libraries you built will be 
dynamic!    ld uses the LD_LIBRARY_PATH to search for code much like any 
other UNIX.
3.  The run time (not to be confused with runtime) linking system is also 
different.  AFAIK Linux uses dlopen/dlclose type stuff.   MacOS X uses 
dyld.  At the time a symbol is called by a running app, dyld will search 
DYLD_LIBRARY_PATH looking for the dynamic library with the same name as 
the one used at link time.  If the file is not there, or it does not 
contain the symbol, your code breaks.

So why bother?  Alex made his life simple by linking everything static. 
Why not do that and avoid all this nastiness?

Unfortunately, making it easy for the original programmer makes life 
harder for the Users.   The whole philosophy of the Mac since day one of 
the project back in 1979 was to make things easier for the user, even if 
that meant a little extra complication for the programmers.  Back then, it 
was serious complication.   One could only build MacOS code on a Lisa and 
move it to the target machine over a serial link.  Debugging had to run 
the same way!   Nowadays, this is not really a problem except for people 
trying to port products like Swarm whose build system contains implicit 
assumptions about the way dynamic libraries build and run.

Why does it make it hard for the users to build static?  Say we discover a 
bug in tcl.  If we compiled static, we have to rebuild tcl, tk, blt, Swarm 
and every bit of code built on them.  Life's too short!

OTOH if we built dynamic we just drop in the patched tcl library and 
everything else inherits the fixes.  Indeed if we used Objective-C, this 
works not just for fixes but for new features.  Now we are cooking.  That 
is why we need eventually to port to GNUstep.  so we can use AppKit.
Enter Frameworks.  If you build needed libraries as frameworks, there is 
no need to worry about DYLD_LIBRARY_PATH.  Frameworks are in a hierarchy: 
those in ~/Library/Frameworks will override those in /Library/Frameworks, 
which will override /System/Library/Frameworks.  Yet there is more!  A 
framework in an application bundle will override everything.  So you could 
package a Swarm application with tcl/tk/blt/hdf5 and the Swarm libraries 
in an bundle and distribute it as a drag and drop install.  Try that on 
Linux!  We are not finished yet.  Since the Frameworks also contain the 
headers, your drag and drop install application is also an SDK for the 
libraries.

This stuff may have a learning curve but it is way cool when you come to 
do real work.  Reusable software and all that.

Finally the usual caveat:  I may have got some of the above wrong, please 
feel free to comment/correct me, but I do feel the fog has lifted for me 
in the last couple of weeks.  I can now see a much clearer way forward.

Bill Northcott

                  ==================================
   Swarm-Support is for discussion of the technical details of the day
   to day usage of Swarm.  For list administration needs (esp.
   [un]subscribing), please send a message to <address@hidden>
   with "help" in the body of the message.