RE: Sawmill's dataspaces and the Hurd's physmem

[yuan Darwin-r62832]

Hi Neal,

        By reading the document of Hurd(VMM), I guess I had a misunderstanding 
about Hurd's approach. Sorry for bothering.

Best Regards,
Darwin

-----Original Message-----
From: Neal H. Walfield [mailto:address@hidden 
Sent: Tuesday, October 18, 2005 9:40 PM
To: yuan Darwin-r62832
Cc: address@hidden; address@hidden
Subject: Re: Sawmill's dataspaces and the Hurd's physmem


At Tue, 18 Oct 2005 19:42:27 +0800,
yuan Darwin-r62832 wrote:
>       I think these 2 approachs are not incompatible. Here are the reasons,
> 
>       1. In Hurd's approach, every application could manage its own 
> physical memory. However, for most of application developers, they 
> don't want to take care of the VM replacement policy. To solve this 
> problem, Hurd has to provide a general VM server to be the pager of 
> this kind of applications. However, as the philosophy of Hurd, should 
> this applications trust this server?

Physical memory management needn't be an all or nothing deal. Certainly, an 
application might wish to completely manage the paging policy and its address 
space layout, however, I tend to think that this is the exception.  And as we 
will provide a POSIX personality, we need to have some sort of default VM 
manager.

The solution that I've opted for is a library based one: a default memory 
management library will, for instance, implement an LRU based eviction scheme 
and require no application input.  For many applications this will be 
appropriate and sufficient.  Those applications wishing to take complete 
control will also be able to completely replace the library.

I think that some applications can provide useful hints in relatively concise 
ways.  A document viewer, for instance xpdf, might want to cache previously 
rendered pages.  It does not make sense to send these to swap if rereading the 
data and rerendering is cheaper.  In this case, the application can attach a 
function to drain the cache to a library provided hook which is called when 
there is memory pressure. I think that this small type of change may offer 
dramatic results. Moreover, if the change is highly isolated (which in this 
case seems feasible), it will be easily accepted upstream.

An even less invasive hint would be to set some environment variables. Clearly 
we wouldn't expect most users to set these but an application's packager could 
based on observed behavior.  In the case of e.g. grep or cat we might want to 
set the read ahead parameter to "very aggressive".

If a developper so desires, a more aggressive, but more invasive approach, can 
also be adopted.  Instead of using malloc and free, the application can use a 
slab allocator.  I think this can only be effectively done if the slab 
allocator participates in the eviction scheme.  Again, this is possible in our 
case with a number of library provided hooks but not for user applications 
running on a traditional Unix-like core.  This can be made backwards compatible 
by having the configure script check for the required mechanisms and if they 
are not available then to redefine slab_alloc and slab_free to malloc and free.

>       2. In Sawmill's DS approach, every task(AS) has a specific thread 
> named "region mapper" to be the pager of other threads. It captures 
> the page fault, then decide to forward it to corresponding server, and 
> get mapped. So from the higher level point of view, these servers are 
> the pagers of the task. If Hurd application should trust that general 
> VM pager, the applications using Sawmill's DS framework should trust 
> these servers as well.

I hope it is now clear that there is no general VM server.


>       3. Relative to Sawmill's approach, Hurd provides a clear & great 
> physical memory server, which makes the whole physical memory of 
> platform could be fairly used by all of the servers & applications.
> 
>       Therefore, we can use Hurd's physmem server as the central 
> controller. Sawmill's DSMs apply physical memory from it. The 
> applications who wanna use Sawmill's approach could still walk on 
> their own way. For some applications who wanna manage their own 
> physical memory, they can apply memory from physmem server directly.

I see a number of problems with SawMill's dataspaces.  The root of this thread 
is the presentation of a potential security flaw in the design of dataspaces.  
(Whether this is important or not depends on the assumed trust model and 
security goals.)  Another is that as far as I can tell paging decisions are 
made towards the root of a dataspace hierarchy and not at the applications 
themselves.

Hopefully it is clear why I've choosen to reject this scheme.

Thanks,
Neal