Re: libports on L4

[Farid Hajji]

> The general type of mach messages consists of data words and references
> to blocks of memory.  This maps one-to-one to L4's IPC, with the memory
> blocks being pages.  Port rights will have to be mimicked.  I think
> that a leaner and more efficient mechanism can be designed to implement
> inter-server communication in a hurd-compliant way.
Agreed. We could in principle tunnel the mach messages into L4 IPC as
opaque data, multiplexing and demultiplexing at each end with some stubs.
Though I doubt that this is the best way to proceed. While we're at
thinking about the general design (of libmom), we should IMHO step back
from mach semantics (like port rights) and consider what we really must
transfer between the different parties of the Hurd.

> > Mach hasn't got the notice of a clan-leader, so we might be able to use
> > that tread as the "proxy" for the messages/ports.
> A clan's chief is a separate task; its (sole?) use is security by
> acting as a censor/gatekeeper.  You can think of it as a proxy, but it
> is one of a different kind.
We should probably consider the whole of the Hurd as running in a single
clan. This way, we get maximum independence of the L4-ABI (think of X.0
in L4Ka right now!). One other advantage is that we run separate OS-
personalities in different clans (sub-hurds too), effectively maintaining
a single entry-point to each personality.

Security stuff should be IMO implemented either in [new]libports or
in the servers themselves. If we think further, one could imagine
accessing some Hurd servers remotely over the net. This would ultimatly
imply, that security can't be guaranteed by a single ukernel alone.
Of course, incoming messages from a remote node would pass through a
clan-leader, if the ukernel version supports clans at all. But should
we really rely on it? Applications running in the same clan would still
be able to RPC servers, so basic security is still needed, even inside
a single clan.

> >> If we were to follow the present implementation of the Hurd, we would
> >> have to start a separate thread for each port; a fileserver would be in
> >> severe danger of overflooding the thread space.  
> > It already is. If you have Hurd running, do a "ps -AT" and you'll see
> > that the ext2fs server does have quite some threads. We *do* want to
> > multiplex more [cp]threads onto one L4 thread, as 128 already is far too
> > little. It's not only the thread-per-port, if I recall correctly you can
> > tell Mach or the ports to spawn a thread with each incomming message (!).
>
> No, it is a function in libports; mach has nothing to do with it.  
> 
> This thread thing looks like the largest hurdle to pass in getting the
> Hurd running on L4.  Possible solutions are:
> 
>  -If a task T1 runs out of threads, create T2 and maps T1's address
>   space to it.  The efficiency of this scheme is subject to scheduling
>   behaviour and can be good if done well (I think, offhand).  The
>   problem now is the depletion of the available tasks, but this is just
>   a variation on the problem of mapping the Hurd's processes to L4
>   tasks.  This method is mentioned in a document on L4; I forgot which
>   one...
This only postpones the problem and nothing really prevents us from
increasing the number of threads-per-task in a particular L4 implementation.
Using multiple tasks only to have more threads would IMO result in
incredibly complex code.

>  -Multiplex user-level threads on L4-threads.  I _think_ this can be
>   done as long as (L4) threads involved in _any_ sort of IPC are left
>   alone; use "worker"threads only.  Again, I have not given this very
>   much thought, so I may be rediculously wrong.
This seems like a feasible solution, iff we can craft up a N:M pthread
library that is stable enough for the heavy use it is going to be put on.
>From a design point of view, I'll favor this approach, but we need
volunteers with the skills of writing such a pthread library (with a
backend for L4 and probably other back ends for other ukernels/guest-os).

Maybe the people at Dresden or Karlsruhe are already working on it?
We could also look at the thread implementation in the OSKit libraries
to get started. Difficult question...

> The following remark may be considered a heresy in this list, but there
> is a school of programmers that regards this prolific use of threads as
> bad design; a comment to this effect was quoted on debian-hurd
> recently, and some of the regular contributors to c.o.l.d.s. wrote in
> the same vein.  So, do we really need this agressive multithreadedness
> to build Hurd-compiant servers?
I like this idea! Too many threads is really one of the main problems
with the Hurd in its current design. Kernel-threads are lightweight,
but not cost-free if you consider performance. Relying on (new) threads
for handling each request does not scale well either. One way to alleviate
this problem is to reuse existing threads by building thread-pools and
using dispatcher threads.

Avoiding proliferation of threads in general is also a good idea. But how
are you going to proceed here?

>    Ducking to avoid rude remarks and ignited Molotov cocktails,
No reason to. Your remark is pertinent and I'd like to (partly) join in.
We could discuss this more thoroughly, once the problem really arises.
It's just that we're still very far from it now.

>      Ad Buijsen            

-Farid.

-- 
Farid Hajji -- Unix Systems and Network Admin | Phone: +49-2131-67-555
Broicherdorfstr. 83, D-41564 Kaarst, Germany  | address@hidden
- - - - - - - - - - - - - - - - - - - - - - - + - - - - - - - - - - - -
Murphy's Law fails only when you try to demonstrate it, and thus succeeds.