Re: ipc security

[Marcus Brinkmann]

At Fri, 22 Oct 2004 01:24:34 +0200,
Bas Wijnen <address@hidden> wrote:
> > Under the premise that you want the IPC to succeed, this is a correct
> > analysis.  Of course, in the Hurd, tasks will be self-paged and thus
> > are in total control over their memory.  So they can mlock() the page.
> > However, this is not strictly true: Except for a small contingent of
> > pages which you can wire completely, the physical memory server will
> > be free to unmap arbitrary pages.  So, either we must use the small
> > protected contingent (which is also used by the pager itself for
> > example), or accept that page faults may happen.
> 
> That's new for me, I expected mere mortal threads to not be allowed to 
> mlock() any page at all.

Well, what is mlock()?  Just a local function call that tells your own
pager not to swap out a page.  Remember that tasks are self-paged.

However, physmem needs to be free to unmap mappings temporarily to
reorganize memory.  It is an open question if we need to be able to
wire down memory in physmem (so that it is not allowed to unmap it at
all, even temporarily).  We certainly need it for DMA, but that is
from a trusted server.  Do we also need it for untrusted user tasks?
If we want to use receive buffer for string items, then forcibly so.
Do we want to use string items in reply messages?  Open question.

> In that case, pagers would have to be paged by 
> some meta-pager (which doesn't do anything else than mapping them back 
> if they were unmapped.)  Of course they may also choose to have their 
> own pager, but at some point an unswappable pager must handle things. 
> This still seems preferable to me, as there would be a (probably quite 
> small) upper limit on the pager size otherwise.  Am I misunderstanding 
> things?

Well, this is stuff that is not laid out in detail yet.  So, your idea
is as much worth as any, until we have studied it in more detail.

> >>(especially if pagefaults may happen in the server's space, because it 
> >>may need to swap some pages in, which of course means others are swapped 
> >>out.)
> > 
> > 
> > I don't understand that.  Above you seem to talk about client's space
> > page faults only (under the premise we had local vs remote xfer
> > timeouts as I proposed them).
> 
> I was thinking of the server trying to transfer a large string to the 
> client.  The client touched all it's receive buffers beforehand, so they 
> are mapped in when the IPC starts.  During the transfer, a server-side 
> page fault occurs, and the server gets some of its string buffers from 
> swap.  Because of this, other pages need to be swapped out.  Under heavy 
> load, that may be the client's receive buffers.  Then the client will 
> page fault, aborting the IPC.

You seem to misunderstand how paging will work.  Memory is never
swapped out without the task which owns the memory requesting it.
This is the whole reason behind being self-paged: You want to let the
task make the decision about paging.  So, under memory pressure (which
may have many reasons), the client can always opt to not swap out the
receive buffers (if you don't have too many of them), but leave them
in the working set.

The other reason why memory may be unmapped from your address space is
that physmem may forcibly unmap it, to copy it to a different location
in memory (to make space for DMA buffers, superpages, etc).  This is
the trouble-some part, and the open question is if we should allow
untrusted tasks to lock down the physical location (and thus the
mapping) to some extent.

In fact, in a more complicated protocol, you could even have physmem
copy the memory, then let the task map the new location (atomically),
and then unmap the old location.  This way you could reorganize memory
without temporarily unmapping the memory.  But this needs the task's
support.  We could make it a rule that the task needs to respond to
that quickly (as with extra fpages).  This idea just occured to me.

The important part to understand here:

1. Swap out decisions are made locally in a task.
2. physmem may still unmap memory without warning.

(number 2. raises obvious questions, which I tried to address in this
and my last mail, maybe re-read it with this in mind).

> Anyway, it wasn't really important for the argument :-)
> 
> >>So either every server which allows string transfers must always allow 
> >>"could you repeat that?"-requests, or there must be a way to tell 
> >>physmem certain pages should not be swapped out.  While having such an 
> >>option may be useful, extensively using it doesn't sound like a good 
> >>idea to me (if only because we may want to make it a restricted request.)
> > 
> > 
> > Well, first and foremost, we already are self-paged, so usually
> > mlock() can have a local implementation that doesn't involve physmem.
> 
> Does this mean that every task can refuse to be swapped out?  I 
> understood that this is currently the case in the Mach-implementation. 
> However, it seems to me like a bad idea.  To prevent denial of service 
> attacks, some kind of quota system will be needed, which likely imposes 
> too strict limits in normal situations.

The number of guaranteed pages that a task may use and reasonably
expect to be available on short notice is negotiated (somehow) in
long-term contracts.  Every task gets a certain memory contingent it
can use freely to manage its working set.

Of course we need a quota system.  And maybe even something simpler
like a free market trading system to encourage tasks to safe memory if
they don't need it etc.  Open questions here how to do it exactly.

> >>I think the best solution is still to use a container from a
> >>container manager.  I'll explain again how that would work,
> >>because I have the feeling I wasn't clear the previous time.
> > 
> > 
> > We do have that, of course, in physmem.  Physmem is trusted by both
> > parties, and acts as the mediator
> > 
> > Your description has the analog problems of IPC security between the
> > container manager and the server.  You don't seem to address that.
> 
> In my (previous) proposal, the server would only do non-string IPC to 
> the client and the container manager.  It would share a container with 
> the container manager.  As far as I understood it, writing in a shared 
> container is not a security issue.  Is that incorrect?  The string 
> transfer would be done between container manager and server, which would 
> have mutual trust.  Anyway, I have a new idea now, which is better than 
> this one :-)

Well, if you think about physmem containers (and I don't think we need
two slightly different container concepts), then you don't need to use
string items at all, you can just map the container and memcpy on the
shared memory (well, it's not exactly that simple, but almost).

> > For sending data, we could also have two interfaces if needed (one
> > using string items and one using containers).  For receiving data,
> > this is less useful.
> 
> I can see that for receiving there is no problem in using strings.  I 
> don't see why containers would not be useful for huge objects.

It is less useful because when receiving data, you may not know how
much you receive, and thus you won't know if string items or
containers should be used.

> > Have you read the paper IPC-Assurance.ps from Shapiro by now?  You
> > really should.  It includes an analysis of the problems and a solution
> > for EROS, the trusted buffer objects (TBO).   Physmem containers
> > provide a comparable feature in our design, although there are some
> > differences in how we envision the typical use (and of course, we
> > don't have kernel support for it).
> 
> I just finished reading it, and it gave me a new idea.  Physmem will act 
> as a string transfer server.  So instead of doing a string IPC to some 
> process, you do an RPC to physmem which will transfer the string.  So 
> physmem, not the kernel, will take care of the copying operation.

Basically, it seems you are proposing that instead of mapping a
container and then writing to the memory mapped, you can fill
containers by sending string items to physmem.

The idea is sound, but there is a problem.  It doesn't work.  :) The
receive buffers have to be set up in physmem before you receive the
IPC.  Any task may send its IPC to physmem, so you don't know for
which container you are receiving a string item next.

This means that you would have to bounce the data in physmem, and copy
it.

The only part that could be implemented is reading data in a container
via string items.  Ie, the server could read from a container using
string items using physmem, without worrying about page faults.  This
might actually be useful.  But writing to containers this way via
physmem is not possible.

> You said containers are very slow.  Where 
> does the overhead come from?  Is it in this scheme as well?

The overhead is that you need several IPCs, and three tasks are
involved, so more context switches etc.  Any scheme involving more
than one IPC between two tasks is going to be inevitably slower.  How
much slower?  Difficult to say at this time.  We'll have to see.

Thanks,
Marcus