capability address space and virtualizing objects

[Neal H. Walfield]

I'm currently working on IPC in Viengoos.  I've decided to mostly
divorce IPC from threads by reifying message buffers.  Thus, instead
of a thread sending a message to another thread, a thread loads a
message into a kernel message buffer and invokes another message
buffer specifying the first as an argument.  If the target message
buffer is not ready (i.e., it is blocked), instead of the thread
blocking, the source message buffer is queued on the target message
buffer.  When the target message buffer becomes unblocked, the message
is delivered.  A message buffer contains a capability slot designating
a thread to optionally activate when a message transfer occurs.

The following diagram shows two threads and message buffer:

                         thread

                       / ^     \
                      v /       v

                   source       dest
                   buffer      buffer

                              /
                             v

                        thread'


The first thread can send a message to the second thread by
configuring the source message buffer appropriately and then calling:

  msg_buf_enqueue (dest, src)

This queues SRC on DEST.

When the message in SRC is delivered to DEST, the thread designated by
SRC is activated, indicating that the message in SRC has been
delivered, and the thread designated by DEST is activated indicating
that a message in DEST has arrived.



This interface poses a problem for virtualization.  One of the goals
of Viengoos is that all interfaces be virtualizable.  This has (so
far) included the ability to fully virtualize kernel objects.
Virtualizing an object is done by way of a message buffer, on which
the same interface is implemented as the object that is being
virtualized.

This means that to virtualize a cappage, it must also be possible to
virtualize cappage indexing.  Imagine that to find the source message
buffer in the above msg_buf_enqueue invocation includes traversing a
virtualized cappage.  That is, after translating some bits of the
address, the kernel encounters a message buffer.  This means that
instead of a kernel-implemented cappage, a message buffer is
encountered while traversing the address space.  Instead of failing,
the kernel should conceptually index the message buffer.  This means
sending it an index message with the residual address.

This is a problem.  The kernel cannot wait for the message buffer to
reply.  It can also not allocate another message buffer.  The reason
is that these techniques violate other Viengoos design principles.
Also, this cannot be reflected to the sender as a "retry using" as the
virtualization would not be transparent.

Note further that the message may contain capabilities, which need to
be looked up in the source address space and saved in the target
address space.

What we'd like then is to iterate over each capability that we need to
lookup and if we encounter such a scenario, save the state in the
source message buffer, invoke the message buffer with the index method
specifying the source buffer as the reply buffer.  And then, when the
reply comes in, recognize that we are in the middle of processing a
message and resume.


Does this approach sound reasonable?  Other comments?


Neal