Re: Let's do some coding :-)

[Marcus Brinkmann]

At Wed, 19 Oct 2005 16:29:19 +0200,
Bas Wijnen <address@hidden> wrote:
> The current discussions on the list are very interesting.  I'm happy to see
> the project getting more attention. :-)  Also, it stimulates me to start
> writing some code for it, and I hope I'm not the only one. :-)
>
> First of all, given the theoretical issues being discussed, any code which is
> written at this point will most likely be discarded when we will go for the
> final solution.  The reason I want to write code anyway is to get a feeling
> for what must be done.

Hi Bas,

I am really happy to hear that you are motivated and want to learn and
help.  This is wonderful, because it can also go the other way:
Difficult technical discussions can be tiresome and demotivating.  So,
your spirit and attitude is really great.

If you feel you must code something to get your hands on the details
and get some real-life experience with the things discussed, by all
means, I don't want to discourage you.  You certainly will learn
_something_, and that is never wrong.  However, if you want to get a
feeling for "what must be done", I don't think that programming a
filesystem or a device driver (this were your examples) would be high
on my personal list of priorities.  Here is why:

I raised the issues of persistence, and the goals we can achieve by
it.  Neal and me believe that these goals are quite important and
worthwhile, and at the current time we are seriously considering if
the Hurd should be persistent.  Now, _if_ we are all convinced that we
want persistence (in whatever fashion), the _last_ thing you want to
write is a filesystem :) You will want to write filesystems, but
somewhere down the road, and there are some interesting technical
problems arising from integrating external stores into a persistent
system, and we want to explore them (it has something to do with
becoming pregnant, ask me if you want to know more).

Device drivers are on the other side of the spectrum.  Everybody seems
to want to write device drivers these days, and I can't find out why
the hell?  Maybe you can help me understand it. ;) I find device
drivers almost border-line boring from a system architecture point of
view.  In fact, they are a burden.  Architectural wise, there are some
interesting problems with persistence and device drivers, and it would
be important to understand this at a deeper level, which I currently
don't.  Maybe this is something that could interest you?  The other
issue is that of sandboxing, which is essential, so there are some
issues around that as well.  Anyway, from a purely pragmatic point of
view, you _really_ don't want to write _all_ device drivers.  A couple
of selected drivers could be rewritten, but just look at the GNU/Linux
source base and ask yourself: Do I want to write all this shit?

Seriously, we are talking about thousands of different, mutually
incompatible, under-specified, incompliant hardware devices that _you
don't have_ and thus _you can't test_.  I am all for pipe-dreaming,
but let's face reality for a shock-second, ok?  Done?  Ok, second is
over.  Now back to pipe dreaming :) If you still want to transform
your home into a garbage ditch for old hardware, let me know.  But
first tell your family, they may want to consult Telsa's diary about
living with a hardware junky
(search for "boxes" at http://zeniv.linux.org.uk/~telsa/Diary/diary.html).

Again, these notes are not meant to discourage you from anything, but
I think you will want to know as much as possible to prioritize your
efforts, so you make them most productive.

The problem we are facing here is that every major design decision we
are making _now_ must be correct down the road in 5-10 years, in fact
much longer _if_ we are successful.  If we _want_ to be successful.
This is a huge responsibility, and it requires great care, and a lot
of studying.  We need a helluvalot of giants, a frigging Army Of
Giants, to stand on their shoulders, if we want to get out of this
alive ;)

This is downright serious.  We are not kidding around here, but
talking about a major operating system with very strong technical
properties that require cutting-the-edge-and-leaving-it-behind modern
programming techniques.  When Linus sat in his student appartment, he
took his i386, pointed at a Unix server, and said: "I want that."  So
he did.  We are pointing into the sky at our pipe dream and say: "We
want that."  The job now is to identify which parts of the pipe dream
belong to the castle in the air, and which parts have a rock-soid
foundation in the latest operating system research.  Then we will take
the latter and add a bit of the former (not too much, though), and try
to build that.

So, to get back to what you wrote.  Your roadmap (writing a RAM-based
filesystem, then adding stuff.  Writing a weak capability system, then
securing it), is exactly what I wanted to do a couple of months ago.
You repeated _exactly_ my plan :) That plan was an OK plan as long as
we didn't reopen the design discussion.  I could also say that it was
good as long as we didn't realize that our design might be broken in
some fundamental ways.

I have a suggestion for you.  It will not immediately lead to usable
code.  You already accepted that your code is for the waste bin, so
you may be OK with that.  Here is what I suggest: Instead of wasting
your time on writing low-level stuff of no real architectural
significance, get some experience at a higher level to get a feeling
for how a good architecture could be used to build applications (I am
not talking about user applications here, at least not only: In the
Hurd, everything is an application).  We have some experience with
writing small programs on top of L4.  Now I would like to see that
repeated with a different OS.

Download EROS or CapROS and build it.  There are instructions for EROS
here:

http://naasking.homeip.net/prog/eros/eros-build.html

I have not tried CapROS (it's the successor of EROS now that it is not
maintained by Jonathan anymore).

Then try to run the hello world example, and after that, write your
own applications.  The EROS concepts are downright weird when you look
at them the first time. Jonathan invented all those funny names:
Keepers (for what we called pagers, or let's say: fault handlers),
Constructors (which we don't have, but we do something _vaguely_
similar with filesystems), and so on.  But hang in there, and try it
out.  Be persistent :)

I wanted to do this a long time, but never got down to it.  I would be
very happy about a small "hacking EROS" introduction guide which just
unconfuses the little confusing things from our perspective, and
allows to write a little client/server application (there is one
example already in the tree to start with).

I think that would be useful, because it would give people the ability
to play with a persistent system with a strong capability system built
in.

So, another test would be to have some semi-long running application,
maybe a very simple text editor, and then kill the vmware process, and
see what happens if you "reboot".  Things like that.

If that sounds interesting to you, then at least from my perspective
this would be more useful than the other stuff you described.
However, let me tell you that I am heavily biased.  Moreover, I
already did a lot of what you said, so I am already "through" that
experience.  In other words: don't trust me :)

Thanks,
Marcus