[RFC PATCH tarfs 0/6] mmap support for tarfs

[Sergey Bugaev]

Hello,

I'm trying to implement mmap support in tarfs.

The reason I need it is I want to run some binaries directly form a tarfs
mount, without unpacking them to disk (ext2fs). ld.so uses mmap to load shared
libraries uncoditionally [0].

So I've been digging through libpager, libnetfs, tarfs, and other code, trying
to understand how things fit together and hook them up. The patch series I'm
attaching implements mmap by lazily creating a pager for each node. The pager
reads and writes data the same way tarfs_read_node () and tarfs_write_node ()
do: namely, by calling into the existing cache implementation.

The most challenging part has been "mmap coherence" (perhaps there is an
established term for this concept that I'm not aware of?). Basically, the file
system has to guarantee that any modificaitons made to a file via io_write ()
(and other similar APIs) are immediately visible through any mappings of the
file, and similarly any changes made through a mapping have to be immediately
visible through io_read (). I peeked at how libdiskfs deals with this (since
this does work as expected with ext2fs in my testing), and it forwards any
reads/writes to pager_memcpy () instead of accessing the underlying data
directly. I've tried to do the same, except that I only forward the calls to
a pager if there is one.

I've written a tiny program to test mmap behavior (I've uploaded its source
here [1]), and used it to verify that my implementation behaves somewhat
sanely. I'm also able to load binaries and shared libraries from tarfs with
this patchset. In fact, I'm able to run code chrooted to tarfs.

Please help me out by commenting on the patches. Does the overall approach look
fine, or am I missing some crucial details? Have I messed up reference
counting? Is there a better way to deal with a held mutex that unlocking it
before calling a function that re-locks it? (Making the mutex recursive,
perhaps? Or extracting the inner part into its own function?)

[0]: https://www.gnu.org/software/hurd/open_issues/libnetfs_io_map.html
[1]: https://paste.gg/p/anonymous/7de6f56e7a1c4babbab17bd6ccc2db46

Sergey Bugaev (6):
  Plumb io_map () through the backend layer
  Link to libpager and initialize it on startup
  Implement basic read-only mmap support
  Implement basic support for writable mappings
  Attempt to implement mmap coherence
  Update TODO and BUGS

 BUGS      |   1 -
 Makefile  |   4 +-
 TODO      |   2 -
 backend.h |   3 ++
 main.c    |  13 +++++
 netfs.c   |  36 +++++++++++---
 pager.c   | 141 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 pager.h   |   9 ++++
 tarfs.c   | 114 ++++++++++++++++++++++++++++++++++++++-----
 tarfs.h   |   2 +
 10 files changed, 302 insertions(+), 23 deletions(-)
 create mode 100644 pager.c
 create mode 100644 pager.h

-- 
2.31.1