Re: why should physmem not know about which frames are extra?

[Neal H. Walfield]

> can you explain why physmem should not know which frames are extra and
> which are guaranteed (I know why it doesn't strictly _need_ to know)?
> If it knew, it would be able to just revoke them, and not go into a
> "notification with some timeout period for the user to reply" mess
> when asking back extra frames.
> 
> As far as I see it (but I made no deep analysis), which frame is
> considered extra and which is guaranteed does not change often.  So I
> don't exactly see why the information shouldn't be recorded in the
> virtual frame info of the container.
> 
> But maybe I am missing something simple.

Extra frames are for caching data which can be easily remanufactured.
The most obvious is data which resides on a back store and when
dropped can be reread.  It is slower, but the data has not been lost.
Another example is data which has been generated algorithmically and
is cheaper to keep in the form of data rather than recreating it every
time it is needed.  For instance, when a user does a readdir, the
libdiskfs calls diskfs_get_directs.  The metadata needs to be massaged
into an appropriate form.  Look at hurd/tmpfs/dir.c for an example.
It might be useful to cache the generated data.  Again, this is an
optimization: we can always go the long way and generate the data by
hand rather than consulting a cache.

When extra frames are in use, they cannot be dropped arbitrarily.
There are two situations which immediately come to mind: when a server
uses extra frames they nearly become guaranteed frames: the data is
locked in core during the operation (although the operation is
typically very short).  For instance, when extracting data from the
cache, the server makes a (COW) copy of the frame for the client.
Copying a frame into a container is not atomic.  Arguably, we could
make it atomic in physmem.  However, this assumes that everyone who
accesses the cache is a Hurd client.  An easy counter example is the
client of a Hurdish nfs server: the nfs server will stream the
contents of an extra frame down a socket.  Servers may also want to
extract data from an extra frame.  Secondly, using the respective
example above, if physmem dropped the frame during the memcpy either a
segfault would occur or special mechanisms would need to be in place.
In that case, we would need to evaluate the cost of those mechanisms
to see if they are feasible and the possible implications.

Extra frames are sometimes converted to guaranteed frames.  For
instance, a page in a cache can be made dirty by writing to it.  Until
the page is synchronized with the copy on backing store, it cannot be
disposed.  The task need only maintain that the number of frames which
it can easily expose of is greater than or equal to the number of
allocated frames.

Finally, just because a frame is designated as extra does not mean
that its cost of recreation is equivalent to all other extra frames.
This is so obvious, I don't think I need to provide examples.  The
only way to properly express the differences is by creating a whole
language and allowing tasks to upload code to the physmem server.
(This is what happened to file usage hints.)  Otherwise, physmem will
make stupid decisions; leaving the logic in the client (until the last
minute) allows for the best decisions about page usage.