Porting the Hurd to VK (Was: running hurd in linux)

[Farid Hajji]

> I think the idea is rather to run Linux with Hurd. For instance you
> could boot something like a MkLinux single server under gnuMach
> and connect it to ethernet1, com1, tty1, etc, and then also have a
> Hurd login on the console.
Porting GNU/Hurd to run on top of a virtual kernel (VK) would provide
exactly this kind of advantage. Basically, you'll be able to boot the
virtual uKernel, either on top of Mach, L4 or even as a server/library
on top of Unix. Then, you could boot the Hurd on top of this VK.

As far as the (new) Hurd is concerned, it wouldn't matter wether the
backend(s) of the VK were Mach, L4 or Unix. Sure, some VK backends
would be much more efficient w.r.t. time and space requirements, but
as long as source code is affected, every part of the Hurd (and of the
Unix emulation layer, currently unfortunatly located in glibc, which
is IMHO a Bad Thing) would use just the VK API.

Personally, I'd also like to hack more on the Hurd, but since I'm
using the BSD flavors (Free-, Open- and NetBSD), Solaris/SPARC as well
as somewhat unusual architectures (e.g. Paragon) which are not
glibc-based, I'm facing one more hurdle to get true
interoperability. This was one of the reasons for me to try to port
the Hurd to a VK that would itself either run on top of a true
microkernel or with the help of a POSIX compliant libc, like glibc but
also other libc's like the *BSDs. The other reason is of course to
port the Hurd directly to L4.

Discussions on l4-hurd and offline showed that the only way to get rid
of the Hurd's dependance on Mach would be to retarget the Hurd to run
on a resonably small (but not too small) subset of some kind of
(micro)kernel. One possible approach would be to assume the following
services from a VK:

* tasks with protected address space
* threads that run inside tasks, preferably accessed through Pthreads API.
* IPC mechanisms between threads of different tasks, user-accessible
  through platform dependent APIs via an IDL-compiler. Only synchroneous
  IPC allowed.
* Per-kernel memory management interface, mostly comparable to Mach's.
  Some VK backends would need to emulate this but I think that this
  emulation layer would not affect performance significantly (if at all).
* Device access through a generic UNIX-/dev-like interface (say:
  read(), write(), ioctl(), interrupt handling hooks). Alternatively a
  device interface closely mapping Mach's (though I'm not sure about
  this yet).

To start the implementation, I'd suggest the following steps.
Please note that this is not a final plan, but work-in-progress and
just a quick brain-dump so that you get a feeling about what I'm
planning to to. Please contribute! Every idea is welcome.

1. Replace MIG with Flick.

   Flick is an IDL-compiler from the Fluke projekt (from the same
   group that wrote OSKit). Flick is flexible, because it separates
   the frontend languages (like MIG, IDL) from the backends (the
   generated stubs like Mach3, IIOP-Corba etc...). As such, the
   easiest step is to use the MIG/Mach3 combination of Flick as a
   replacement for MIG in the first phase.

   The second phase would be to replace the MIG *.defs with true CORBA
   compliant *.idl files. Flick, in the IDL/Mach3 mode would then
   transform the new *.idl files into Mach3 msg_send()/msg_recv() in
   the (hopefully) same way as MIG currently does.

   Depending on the kind of VK-backend, other IDL compilers could be
   used. As long as we stick to standard CORBA-IDL (instead of *.defs),
   we've got a lot of possible choices. Using Mach, Flick seems the
   only alternative right now. Using L4, we could use IDL4 from Karlsruhe
   or a new IDL compiler from Dresden. Using Unix, we could use any
   CORBA-IDL compliant IDL-compiler.

   If we opt for C++ as the main language for IPC parts, we can use
   MICO, omniORB and ACE+TAO (among others) in the VK-over-Unix case.
   CORBA compilers that generate C bindings are less widespread, but
   as a first shot, we could always used ORBit. Personally, I'd opt
   for C++ so that we get maximum flexibility and clarity.

2. Write a Pthreads wrapper around C-Threads.

   A (small) part of the Hurd sources generates Mach threads using the
   C-Threads library. Because C-Threads are not widely available in
   non-mach environments, the sources of Mach should use (a very small
   subset of) Pthreads API instead. In the Mach case, a small
   Pthreads-on-top-of-C-Threads wrapper would suffice.

3. Reimplement the Servers' access libraries.

   The Hurd's Servers are currently accessed either through glibc's
   sysdeps, or through specialized libraries (like libdiskfs, etc...).
   To preserve the Hurd's external interfaces, the backends of those
   libraries (the parts that currently RPC the servers) need to be
   replaced by IDL-generated stubs. As a side effect, the RPC parts
   would have to be moved out of glibc! That's not the place for them,
   despite what glibc maintainers assume (IMO).

   At the other side, the servers' access libraries (mostly the hostmux
   and usermux parts) would need to be attached to the IDL-generated
   skeletons).

4. Reliable credentials:

   Currently, the Hurd's security relies on Mach-issued and ensured
   credentials (port-rights). The first step here would be to implement
   a port-rights server that would emulate Mach's port right semantics,
   at least as far as the Hurd is concerned. This server would be
   contacted in exactly the same way as other Hurd servers (that is,
   through IDL and probably through a helper-library).

   A later step would probably replace this central credentials server
   with a distributed scheme, since this server is a single point of
   failure and a possible bottleneck w.r.t. performance. It is not
   sure wether a distributed scheme or a centralized server are better
   though...

5. Devices:

   This is a tough problem! To keep up with new device drivers from the
   Linux and *BSD worlds, we need to find a way how to "plug" those
   drivers inside VK. One approach can be seen in OSKit. Other ways
   are possible as well. Currently, I think that the easiest way to
   cannibalize foreign drivers would be to provide an emulation layer
   that realizes Linux and 4.4BSD, at least as far as the drivers'
   upcalls are concerned. I really don't like this, but it could be
   * a kludge to reuse new drivers (as long as they are "friendly"),
   * a cheap way to add binary compatibility in a way similar to
     *BSD's Linux, Lites/Mach BSD compatibility.

   People working on oskit-mach have much more experience w.r.t. this
   particular issue. We'll have to take a look at their work before
   doing anything at all here.

If you're interested and looking for a task of things to do, please
write to address@hidden It is impossible to do this alone in a
reasonable time-span ;-).

Regards,

-Farid.

-- 
Farid Hajji -- Unix Systems and Network Admin | Phone: +49-2131-67-555
Broicherdorfstr. 83, D-41564 Kaarst, Germany  | address@hidden
- - - - - - - - - - - - - - - - - - - - - - - + - - - - - - - - - - - -
One OS To Rule Them All And In The Darkness Bind Them... --Bill Gates.