A generic IDL for the Hurd (was: A comment about changing kernels)

[Matthieu Lemerre]

"Jonathan S. Shapiro" <address@hidden> writes:

> The one thing I *will* say about this decision is that it is a "one way"
> decision. Once a significant body of code is committed to an older,
> insecure microkernel API, it will become progressively more difficult to
> change that decision later.
>
> Since either the old or the new Hurd could be implemented on the more
> secure foundation, this *may* be a reason to use one of the newer
> kernels in either case, *provided* they become available to be used soon
> enough.

This was actually the goal of my rewrite of the communication
mechanisms of the Hurd on L4: writing servers, acting as if the secure
mechanisms were available, to ease porting later.

I then realized that we couldn't abstract the communication
abstractions by writing C-code only.  We also needed an IDL.

I then began writing a generic interface generator.  I don't have much
experience in IDL (I know only mig and IDL4), but I tried to design it
based on the following observations:

*  The actual IDLs don't allow enough description of the interface.  For
   instance, we cannot indicate that a RPC might block.  This make
   impossible, for instance, to delay the choice we have on asynchronous
   calls (which is either we wait for the server, or we wait for a
   notification server.  Which may be a bad idea, but that's not the
   point).
   
*  To be generic, generic types must exist.  For instance, a capability
   is a generic type: it can be either
   -A couple (thread id, handle) in L4 X.2
   -A communication endpoint in L4.sec
   -A couple (cap ID, endpoint) in L4ng
   -A local handle? in EROS/Coyotos
   -A port on Mach

   I thus designed code to allow definition of generic types.  So, for
   each definition, you can add hooks checking if a type contains a
   capability (in case of a struct), of is an alias for a capability
   type, etc...

   Another generic type could be for instance memory: a client often has
   to provide memory to the server to be served.  This memory can take
   different forms:
   
   -A capability on EROS (to a space bank) or the Hurd/L4 (to a container)
   -A fpage on L4
   -A vm_address_t on Mach

*  Auto allocation/deallocation: when writing clients, we often
   encounter code of that type:

   allocate(provided_object)
   call_rpc(called_object, provided_object)
   deallocate(provided_object)
   
   I encountered that when provided_object was a capability box (a slot
   in the cap server in which the server sent a capability), but this
   would also apply when doing IO for instance.
   
   We would thus want the provided objects to be automatically
   allocated/deallocated.  mig provided this type of automatic
   deallocation, but this had to be written as part of the IDL
   definition.
   
   I'm thinking of a more flexible solution : when an application wants
   to use an IDL, it can define an ACF file. (application configuration
   file).  The ACF derivates from the IDL and can add or modify certain
   attributes (such as allocation/deallocation, it could also rename the
   functions generated, so it could for instance generate two functions:
   one with allocation one without, or use an argument as in Mach).
   
   An ACF could derivate from an ACF, and so on. You could also use the
   original IDL without providing additional ACF (in fact, ACF is just
   another name for "derivative IDL").

   I know that capIDL is hierarchic, maybe this is what it already does?
   I didn't had time to check yet.

*  This were some of the core ideas.  On the implementation side, I
   choose to parse DCE IDL (because its syntax is more suitable for C
   programming, and that's what we do) (but it's replacable), to perform
   some analysis (like type and definition dependencies), and to run some
   hooks for every definition (some of then output the code).
    
   We aren't forced to output C code: I planned to output both C code and
   IDL4 definitions (thus taking advantage of the high performance
   generated code of IDL4).
   
   For now, it can parse DCE, perform some analysis and run different
   hooks.  I did not write the hooks to output all the code yet.  I
   wrote all that quite a while ago, and had no time to pursue (I
   don't even have time to read the mailing list!)
   
   Oh, and I didn't mention it.  I chose to write this in scheme (for its
   abilities to deal with symbols, perform grammar transformations, and
   run hooks).  And because it's a good language of RAD so I could test
   the prototype.

*  Why did I write this?  I wanted to know if this was feasible, and I
   now think that it is.  Marcus kept telling me we can't be
   microkernel agnostic (and I think this is true, there are many
   other issues like how paging is handled, etc), but at least the
   possibility of beeing less dependant of the RPC stub code (which we
   wrote at hand) is a good thing.  I think this is a mean to be able
   to ease porting when a microkernel shift occurs (either minor or
   major, and such a shift will always occur).
   
   I'm sure that all EROS servers won't be rewritten when switching to
   Coyotos; some can be rewritten without major modification.
   
   Maybe we could even extend it to be able to test some servers more
   easily (for instance, using UNIX pipes.)  Obviously every server is
   dependant of its complete environment (and this is, I think, even
   more true on EROS), but maybe we could have some basic framework to
   perform (at least basic) unit tests.

   At least, I think most of the definitions are independant of the
   microkernel; if generated code can't be ported, at least we could
   keep the definitions.  And the most precise (i.e. with the most
   attributes) definitions are the best help to understand them.

I'm just wondering if my approach is a complete mistake, and if you
have any comments.  At least it could be used to perform basic
conversion between .defs files and capIDL ones.