Re: synchronous RPCs vs. asynchronous RPCs

[Niels Möller]

Marcus Brinkmann <address@hidden> writes:

> "We should only do synchronous RPCs, and implement all asynchronous RPCs as
>  synchronous RPCs performed by a helper thread."

I agree that syncronous calls should be the usual way of doing rpc.

> This applies to select()/poll(), asynchronous msync() and other operations
> like that.

I don't know about msync, for I care about poll. I think it is
important to be able to poll/select on thousands of objects, without
using thousands of threads.

One way of doing poll may be as follows, where the main point is that
a single thread receives notifications form multiple servers.

1. First set up a receiver thread R, which starts an rpc receive from
   any thread. Wait until it is in the blocking state (that may need
   some syncronization, but I'm not caring about that now).

2. In the original thread, send an rpc to on each polled object, with
   the following meaning: Say if it is possible to do i/o on the
   object immediately, and if it can't, send a message to the thread R
   when i/o becomes possible.

3. If no i/o was possible immediately (which is the interesting case),
   we'll wait until R receives a message, or the timeout expires. R
   will be able to reliably receive at least one message from one of
   the servers.

4. When R has received a message, saying that i/o is possible on one
   of the objects, we again loop over all the objects, asking them if
   i/o is possible. From the collection of replies, we construct the
   values to be returned by poll().

In (2), we either have to include the thread number of R in the
message, or fake the sender thread id in some clever way (I'm not sure
exactly how that feature works in L4).

In (3) there's a DoS issue, an evil thread can send loads of messages
to R so that it unblocks, uses a few cycles to discards the spurious
message, but misses the real messages from the servers.

Sending rpc:s to all the servers seems a little expensive, it may be
possible to poll on n file descriptors with fewer than O(n) rpc calls
on average, like this: We spawn R, and ask all the servers to always
notify R when i/o becomes possible (regardless if we're blocking in
poll at that moment). R will get most of these notifications and keep
state.

In poll, we first check if R already knows about some objects on which
we can perform i/o, if so, poll can return immediately we no extra
rpcs. Only when poll is about to block (or after a sequence of X calls
that didn't block) it goes through the loop (2) to make sure that it
will recover from missed notifications.

Does that make sense? Or do you have some completely different plans
for poll?

One should use similar mechanisms for all of poll, file and directory
notifications, waitpid, and the pthread function to wait for
particular signals (don't remember the name), so that one can use a
*single* hurd operation to wait for i/o on fd 7, changes to the directory
/bar, death of process 17, or SIGWINCH, whichever occurs first.

I think one can aim for a design with *one* notification protocol (ok,
perhaps two, if task death notifications are special enough to have
its own protocol), and make everything else syncronous.

Regards,
/Niels