Re: loader question.

[Marcus Brinkmann]

Hi,

I should say one thing up-front: I have no idea what you mean if you
say "the loader".  In our design, there can be many loaders, and the
"loader" is in general a mixture of the parent task, the filesystem
providing the binary data, and the new task being executed.

At Tue, 17 Aug 2004 18:23:15 -0400,
Rian Hunter <address@hidden> wrote:
> Since all tasks will be self-paged (i don't know if this has changed)
> given physical memory addresses, how will program loading be handled?
> My issues are with the conflicts between the memory already mapped
> virtually for the program, with physical memory pages requested from the
> memory server maybe having the same address (although this is unlikely).
> Another issue is if the pages used by the executing text (+data) will
> count as pages own by the task or not, if it is the task's
> responsibility to handle it's own code pages.

Memory mapped from the physical memory server does not need to be
mapped 1:1, ie at their physical address, into the task.  In fact, it
will usually not be mapped at their physical address (the task does
not even know the physical address).  All memory mapped into a task is
mapped to virtual addresses.  The task can usually decide to which
address it wants to map the physical memory provided by the physical
memory server (with some provisions that allow the physmem server to
reorganize mapped memory, ie by splitting a mapped region up into
smaller chunks).

Data mapped from files (be it the program data, or any other data
mapped in via mmap), must be paid for by each task individually
(although the actual memory can be shared behind the scenes with
support of the filesystem), and yes, the task must manage this memory
itself, just as any other memory.

> Also with the self-paging: Assuming a POSIX program is run, completely
> oblivious of the advantages it is offered with self-paging, how would a
> malloc be handled.  Would there be a seperate simple pager thread,
> specific for each thread, started in the task to handle the malloc
> request and to initially request memory from the physical memory server.
> If so, this thread would have to be transparently started and loaded for
> each POSIX program, adding the issue of notifying the loader which
> programs are POSIX or not.

Yes, exactly as you say.  The execution of a new task is a complicated
process.  However, the loader exists mainly in the program being
started.  This means that the loader is part of the POSIX program, and
no explicit notification by some other entity is necessary.  The
loader in this sense is of course the linker that comes with glibc,
which will be linked to all POSIX applications.  There is a lot of
other behind-the-scenes stuff that comes with providing programs a
seamless POSIX environment, malloc is just a small part.

We will not write our pager for POSIX programs specifically, though.
There will be a pager interface that is used by glibc, and then you
can link different pagers into your application, depending on your
needs.  Thus, you could experiment with different caching policies
even for POSIX programs that are unaware of them.

But, also remember that POSIX has madvise(), which can be useful in
this context.  madvise() would then notify the internal
behind-the-scene pager about the desired policy for a given mapping.

> If POSIX programs are run not with local pager threads, but with a
> global pager task(server) for the POSIX personality, then some UNIX-like
> VM would have to be implemented in that server. That server of course,
> would use the real physical memory server. Also this would simplify
> things as malloc (which would only be called in POSIX programs) would
> always know to use this server, instead of some arbitrary local thread.

It's true.  It's what Mach does.  You could write a global VM, but
that is inefficient by design, because this global VM would have too
little information about the way the memory is used.

Thanks,
Marcus