Re: New thoughts about deva/fabrica

[olafBuddenhagen]

Hi,

Bas already explained all important points (thanks Bas -- it's quite
reassuring to see someone shares my ideas :-) ), probably better than I
could ever do.

However, there are a few points I would like to elaborate on.

> After I read antrik's proposal about "posix level device drivers" I
> thought that was a great idea. I spent the last 3 days reading mailing
> list archives, and now I'm unsure it is. Actually, most of the
> advantages listed there can be implememted in the original ddf
> proposal by Peter De Schrijver and Daniel Wagner.

Sorry, but that is just not true. What's true is that *part* of the
advantages *could* *theoretically* be *emulated* on top of deva, from
the *user's* point of view. Oh well, only it won't happen.

Making the drivers more or less look like ordinary applications, with
all the implications connected to that, would require boatloads of
wrappers and black magic. A hell of a task to do,
reimplementing/integrating all those Hurd interfaces. Considering that
effectively it means an emulation, doing it really well quite likely
would mean more work than the Hurd itself! Nobody will take the trouble
to implement even a fraction of it. Really, those proposing this do not
seem to be remotely aware of the complexity of such an endeavor.

Then, there are those features that even in theory are a virtual
impossibility to emulate, like running drivers directly as ordinary
programs (as opposed to using magic enchantations that load the actual
drivers by some obsucre mechanism in the background); using standard
input/output streams; removing the barrier between drivers and higher
level applications...

And none of the above even mentiones stuff beyond the user's point of
view.

So how many allegedly non-unique advantages we have left, that would
*realistically* be implemented on top of deva?...

Seriously, why do all the additional work first to implement a special
interface for the device drivers, and then even more work with really
fat layers on top of that to make it fit into the system and look like
other stuff? Why not just skip that crap and use the native POSIX/Hurd
interfaces directly?

> Second, while readin' the mailing list it tourned out for me that
> there're lots of disadvantages we have with posix level drivers, and
> not the smallest one is speed. 

Pray tell me, which are the others?

As for performance: No question, this is an important concern. But is it
a problem, too? Let's take a closer look.

I can see basially three important places where the POSIX level approach
touches the drivers:

1. Startup. I guess you'll agree there are no performance problems to be
expected here.

2. Resource management. Of course this is higly performance relevant --
but in the sense that smart resource allocation is important. The actual
runtime communication is quite little, so the interface plays no role.

(Actually, this is probably a serious downside of the deva/fabrica
proposal. For those that aren't tracking it: Neal is working on a nice
resource management framework, in which applications, in a hurdish
manner, announce their own resource needs, and the manager fills them as
good as possible while keeping fairness, using a market-based approach.
Being among the most performance-relevant tasks in the system, with
real-time behaviour and all, the drivers obviously need to make
extensive use of the possibilities offered here. However, this resource
management interfaces will be inherently Hurd-specific (or at least I
see no feasable way to make it generic), thus defeating the whole idea
of a system independant driver framework...)

3. RPC. This is actually the only point where overhead can be really
performance-relevant. Luckily, this is also the one least affected by my
proposal.

While I strongly encourage using filesystem semantics unless there are
good reasons not to, there is nothing in my proposal making them
mandatory for all interfaces. I explicitly mention that while the
majority of drivers probably won't experience any relevant performance
hit, for the few ones that would, it is perfectly possible, valid, and
desirable to implement shortcuts in the form of optimized custom RPC
protocols.

In fact, I see a good chance that we might find some shortcuts useful
enough to be applied outside the driver framework. If so, they could be
implemented as generic extensions to the basic filesystem interfaces,
like mmap and some others are. But that's just an option I like to muse
about :-)

So, is performance really a problem with POSIX level drivers? I don't
think so.

> And fourth, there's a pragmatic reason: a microkernel allows to run
> multiple OS in parallel. And we shouldn't skip this great advantage.

Like Bas, I feel a good and simple driver framework to be much higher
priority than the obscure option of running other OSes in parallel on
the same kernel.

But then, it's not a fail-pass situation anyways.

As you noticed yourself, the deva/fabrica proposal is about drivers
portable among different L4-based systems (which is just overhead not
worth the effort IMHO), but not about running several systems *in
parallel*. (Most likely, it even hinders that.)

Specifically, the idea was to rip all management and access stuff out of
the drivers, and move it to a system specific server (over a narrow
well-defined interface), which forwards that to the actual system
services. (Ironically, this looks much like Shapiro's POSIX emulation
approach, as opposed to the Hurd approach...)

For running multiple systems in parallel, you need more: Not only
provide management services to the drivers, but also sharing of the
drivers as well as of basic resources among the OSes running on top.
There are two possible approaches for that, which are mostly equivalent:
Either you put all that stuff into the driver framework itself, making
it self-contained, and base the actual systems on that. Or you let one
of the main systems do the work.

They are equivalent, because you do not actually run several OSes in
parallel anyways. What you really do is running a single OS, composed of
some shared services, and a number of independant subsystems on top.
It's just a matter of definition whether you consider the shared core a
component on its own, or attribute it to one of the main systems.

I prefer the second variant, because it reflects the fact that in our
case the core and driver framework actually belong to the Hurd; and more
importantly, it reflects the fact that the driver sharing needn't be
symmetrical.

That is to say: It is perfectly possible (and reasonable) to make the
drivers run as part of the Hurd, fully integrated and using native
interfaces as fit us; and then export the driver functionality to the
neighbour systems. (Either exporting the native interfaces directly,
requiring the other systems to implement them, or making the other
systems' lifes easier by wrapping the driver functionality in a more
generic interface, and exporting that.)

The important thing here is, that it's completely irrelevant whether we
do the export on top of some special driver interface, or our normal
Hurd/POSIX interfaces.

On a completely different note, there are some considerations to make
deva a library instead of a server. While this may sound like a simple
substitution, it really makes it much closer to the POSIX driver
proposal than to the original approach. The one remaining fundamental
difference is that instead of accessing the Hurd facilities through libc
using POSIX, it would access them through the deva library using some
special deva interfaces. I wonder, what for?

People, please, you need to get away from the absurd notion that drivers
are some aliens, inherently requiring special handling completely
different from the rest of the system. The more I think about these
issues, the more the POSIX level approach turns out to be the obvious,
natural, straightforward one. There is really no need for justification
of the POSIX level driver proposal; it's anyone suggesting a different
approach that needs justification.

-antrik-