On Compatibility

[Jonathan S. Shapiro]

Alfred has raised some valid points about the importance of
compatibility. However, there are really two different issues to
consider:

  1. There is a large investment in POSIX-dependent GNU code
     that we do not want to give up.

  2. There is a large user base of existing applications, and
     we need a way to support and transition these applications.

For a long time, I was very concerned about point (1), which is the
point that Alfred is raising. Because of the infeasibility of altering
this very large body of code, I was concerned that an EROS-like system
would never succeed. Alfred's concern is well-motivated and entirely
valid.

His point is valid in spite of the fact that the *existence* of the GNU
project is a counter-example showing that it *is* possible to build a
system from scratch!

However, I think that Alfred is starting from the middle instead of the
beginning. I would like to explain where I think that this discussion
*should* start, and then explain why I am not convinced that POSIX
support is not going to be a big problem.


   IT IS NOT A GOAL TO SUPPORT POSIX FOR ITS OWN SAKE!

The desire to support POSIX is a consequence, not an axiom. The
important goal is to support *users*. In order to support these users,
there are certain bodies of software that we must preserve, modify, or
replace. Some of this software relies on portions of the POSIX
interface, which motivates us to look at POSIX compatibility. However:

One of the major changes that has occurred since 1984 (when the GNU OS
project started) is the commodity introduction of graphic displays and
the move to GUIs for end-user applications. I personally had on my desk
at Bell Labs one of the very first batch of smart bitmap displays in
1981. I have watched the transition with some fascination.

Until very recently, the UNIX world has constructed software by
developers and for developers. The use of graphics by UNIX development
tools remains pathetic even today. Among all of the tools that I use
daily in software development on UNIX, there is no tool that will not
work very happily on a VT100 terminal today. These programs are all
command line oriented, and they are heavily POSIX dependent, but they
have no "exciting" dependencies.

Then there is the *other* class of UNIX software: things like evolution,
and firefox, and OpenOffice, and various "applications". These are the
things that the typical end user sees. Nearly all of these are
cross-platform tools, and *because* they are cross-platform tools they
have very little dependency on POSIX. Instead, they are written to
cross-platform toolkits such as Qt, GTK, and a few others.

So if we are going to look at portability issues, there are actually two
different interface layers we need to look at:

  POSIX, which has become the de facto portability layer for command
  line tools. The overwhelming majority of the FSF/GNU tools fall into
  this category.

  GTK, Qt, and a small list of other toolkits, which are the de facto
  portability layers for end-user applications. This is the stuff that
  makes real users run your system.

I think that the Hurd needs to make developers feel at home, but
ultimately it also has to be useful for end users.

As part of a contract for the Air Force, my lab started a GTK port to
EROS. Along with many other research contracts, this contract was
terminated early when the Bush administration decided to go into Iraq,
but we got far enough to determine several things:

  1. Porting GTK is a pain in the ass, but making an EROS-native
     GTK isn't really a big deal.

  2. The hardest part in porting GTK is porting autoconf, because
     autoconf is *incredibly* POSIX dependent.

The good news is that once we bring up a POSIX emulation environment, a
native port of GTK is merely work, and then a bunch of applications
should come up with minimal change.

  NOT ALL OF POSIX IS REQUIRED

When we look at the POSIX-based applications, they basically divide into
two classes:

  1. Developer tools, and
  2. Tools that manipulate privilege in unusual ways.

The first category can all run happily in a POSIX compatibility box.
This approach worked pretty well for NeXT, and it is now working fairly
well for Apple (in the sense that the end user is unaware that POSIX is
under the hood unless they go looking for it). You actually do not need
very much of POSIX to make this category work -- a fact that has been
demonstrated in several research systems. You simply don't need very
much of POSIX to support GCC *or* emacs. For example, you do not really
need uid's at all.

The second category is simply broken, and really *needs* to be replaced
if we are going to talk about security in any sensible way. So we are
either going to replace this stuff or give up on security. Maybe the
answer should be to give up on security, but if we are going to do this
we should do it with our eyes open.

  THE PART OF POSIX WE NEED IS NOT A BIG DEAL

The portion of POSIX that we actually need in order to successfully port
the FSF/GNU tools really isn't that big a deal at all. It can be done
compatibly, and we need to do it. One way to approach it is illustrated
by Mark Seaborn's work on PLASH.

In any case, I propose that the correct metric of compatibility is to
determine what applications we need to bring forward, and use these to
decide how much or how little of POSIX we actually require.

Separately, we will want to do a full POSIX compatibility box for
execution of legacy applications.

shap