Re: Getting Started with Hurd-L4

[Marcus Brinkmann]

At Thu, 21 Oct 2004 19:27:40 +0100,
Sam Mason <address@hidden> wrote:
> 
> Marcus Brinkmann wrote:
> >Well, I will write a smallish task server any day now, which should be
> >good enough to create new tasks and allocate threads.  The physmem
> >server definitely needs more than the current dummy container,
> >although it turns out good enoough to run the task server and allocate
> >some anonymous memory (with a frightening dumb interface).
> 
> It'll be interesting to see what the difference is!  Everything is
> pretty new at the moment, so I don't really know what's
> incomplete/broken at the moment.

Don't expect too much.  We have done a lot of work, but most stuff
still exists in our heads only and is not realized in code.  Still,
that is the difficult part :) as we are not aiming for a toy OS but
for the real thing (which means to anticipate a lot of stuff that is
annoyingly difficult to get right in every detail).

So, what is there is mostly framework to get started.  In particular,
my aim is to get enough code running to allow others to jump in and
hack on an isolated part without worrying about other parts (ie,
enough should be in place to run and test isolated components).
 
> >Then we need some fake/dummy device access server [. . .]  That's
> >another couple of days easy hacking.
> 
> Assuming you know where to put it!

Well, sure.
 
> >At that point, user level stuff can start (porting the C library, etc).
> 
> I think my wording of "user level" was probably a bit off!  Not sure
> how to describe what I mean though.  I guess I'll just hang around a
> bit and see how everything moves on!  I was expecting things like the
> C library to be a *long* way of yet though.

Well, there are usually two definitions: Everything outside the kernel
(user space), and everything outside the system code (untrusted user
space).

As we use L4, everything we do is user space (except for the rare case
where we find a bug or flaw in L4, etc).  So I was assuming you meant
the untrusted user space.

You mumbled something like running a shell would be far off.  I don't
disagree.  However, a minimal shell that does load an executable from
a ram disk and creates a task from it will be possible quite early.
That won't be a POSIX shell of course. ;)

But I don't actually think much in those terms.  My perception of the
task ahead is orthogonal to what programs will run.  I think in chunks
like: thread cancellation, RPC stub generator, signal handling,
fork(), shared memory, virtual memory management, paging, dynamic
linking.  Things like that.  I don't know yet how much of these things
must be in place for even to be able to hope that bash compiles.  But
if we are lucky, we can fake enough to compile some simple programs
early (even if they will fail to work properly in some circumstances).

By the way, the C library is not a long way off.  In fact, I already
have an older version of glibc (a year ago or so) frobbed so much that
I could compile it on a bare-bone L4 target and use it.  I had to hack
it somewhat to avoid using malloc during the startup, though, but the
result linked and I could use some glibc functions which don't require
much (or any) system support.  Originally I wanted to use them for the
rootservers, but that idea is kinda off the agenda for now (and
likely forever).

The nice thing about glibc is that you can relatively easy compile it
without a complete port, and then implement the missing pieces one by
one.

> I know that it looks like quite a nice design now - which is
> probably why I'm getting a bit more interested.  It also seems to
> be converging significantly with my own ideas on how things should
> be designed and I was hoping to have some interesting banter with
> people about it.

We are open to new ideas, although we often reject very important
topics to limit our scope.  Ie, we don't do (hard) real time,
orthogonal persistency, exokernel like access etc, but we do try to
make those features possible to co-exist or be developed later.  The
more applicable your ideas are to what we are doing, the better.

If you don't get much of a response to any particular idea, it's
probably not because it's not interesting or worthwhile, but just
because it is not what we are putting our focus on.  Nevertheless, we
invite you to try.

> >I'd recommend
> >to read the design notes in the doc/hurd-on-l4.pdf document.
> 
> I've already been reading that; I've also been trying to get my head
> around the code, and I've been dipping into mailing list archives
> a bit to try and get a bit of an idea of how everything fits together.

Much of the design is not realized in code.  However, the parts that
are realized in code are usually more detailed and accurate than the
design doc.  The only really worthy part to look at (if you are
interested in this sub-topic) is libhurd-cap-server/, which contains
the server side of the cap implementation and is near to feature
complete (a crucial part, cap transfer, is missing but I have the
implementation details almost done inmy head for that missing piece).

The only other part that has some merit is the wortel/startup.c and
wortel/wortel.c:start_task() code which shows how to spawn a new
program (ie, the exec protocol).  That is still under development,
though.

The rest (except for the other support libraries, like libhurd-slab
and libhurd-ihash, and libpthread) is basically bootstrap junk.
 
> >The TODO list is just the "working set" for
> >things to keep track of so they don't get lost.  The real stuff is in
> >our brains and in that document.  Ask questions, you'll need the answers ;)
> 
> OK, I'll carry on reading a bit so I don't make too much of a fool of
> myself, and I'll see what gets committed to CVS over the next few
> days.

Don't hesitate to comment on the design.  Defending it helps us in
understanding what we do and clarifying the documentation.

Thanks,
Marcus