Re: bug in task server startup code

[Marcus Brinkmann]

At Thu, 07 Oct 2004 21:49:46 +0200,
Bas Wijnen <address@hidden> wrote:
> > The memory is mapped in fpages, maybe mapping one fpage, the one with
> > the addresses you accessed, worked, and another one failed?  Still,
> > you would not expect to see wrong data, and in your other mail you say
> > the offset was actually 0x1000 off or so, which would indicate to me a
> > bug in the ELF loader/startup mapping stuff.
> 
> I thought of that, too.  The code is at its correct position in wortel, 
> but there seems to be something wrong with the mapping.  However, I have 
> seen it work as expected, too, which for me points to a buffer overflow 
> (which brought me to valgrind ;-) )

There is also a lot of bit frobbing involved in calculating the
fpages.  It might be off by one for some cases or so.  I am sure I
will hit the bug soon enough when I change the task server (ie, when
it grows).
 
> > Mapping memory is to be considered a restricted operation.  We can
> > enforce that by using redirectors.
> 
> Eh, I don't see the problem.  What's wrong with threads mapping their 
> memory to other threads?

It consumes kernel memory.

Imagine thread A in task A and thread B in task B.  Both threads
reside in different address spaces.  Now you map a single page in A to
B.  Then B maps the page back to A, but at the next page address.
Then A does the same to B.  And so forth.  You have only a single page
of memory, but in a 3 GB address space you can install a helluvalotta
mappings from that single page of memory, sucking up all the kernel
memory reserved for page tables.

Same reason why you can't map a page from one address to another in
the same address space (grant operations will work, though).

Actually, recursive mappings are what the L4 people are very
interested in, so it is enabled by default.  However, I think from the
L4 peoples point of view it would be only used directly by trusted
system servers, while the user code would be encapsulated in a
redirector (which you can put into a smallspace to avoid page table
switches at IPC).

(The Hurd design doesn't even use recursive mappings - all mappings
are done flat by physmem.)

> > If it is not clear to you why it is always the client's fault, I can
> > explain further.  Let me know which case you are interested in (simple
> > IPC, string items, map items).
> 
> One thing about string items.  I recall you saying (in an e-mail or 
> comments in the code, I don't remember) that they may be supported at 
> some point.  That surprised me, because the reference manual 
> specifically states that page faults during a string ipc can lock both 
> sender and receiver (with a malicious pager on either side.)  I expected 
> that to be enough reason not to use them in the hurd, because there is 
> nu mutual trust (other servers can use them of course, but not hurd 
> servers.)

String items can be supported, but one must be careful.

The idea is that you allow page faults on the server side to happen,
but not on the client side[1].  The client must then lock down all
memory needed for string items to or from the server.  This is easier
said than done, especially if you don't know up-front how long the
reply is going to be.  There is a whole paper on this issue (by
someone else, in fact, the paper that made me start to worry about
this in detail), and I have written about it in length before.  Look
at the xfer timeout parameter of the IPC system call for how to avoid
being blocked when page faults occur.

It's thus a bit complicated to use string items, but they are an
important optimization.  You can't just always pass container handles
to mappable memory around, for example for every filename you want to
lookup.  We have to decide how to use this on a case-by-case basis.

Also, there are cases where you don't need to lock the memory, for
example when reading directory entries, or even when reading stream
data.  You can just process the data you got until the first page
fault happened, and then repeat the operation to get more data.  This
however might violate atomicity requirements (if any).  But consider
POSIX and interrupted read()s and signals, and think of a pagefault as
a kind of signal, and you are getting there.

Thanks,
Marcus

[1] I am a bit unsure about this.  Letting page-faults happen in the
server when receiving data might be undesirable from a performance
point of view, as no other RPCs can be processed during the page
fault.  The only save page fault situation may be page faults in the
server when replying to messages, because then it is the worker thread
being blocked.

This is not very relevant if the server only ever allows small string
items (a single page, for example), but may be important if you want
to allow huge string items, unless you somehow prepare for it, for
example by preallocating superpages.