Re: [Nmh-workers] What are and what should be the qualifications for a current nmh user

[Mike O'Dell]

since the beginning of MH there has been a HUGE fundamental
change in the structure of Email. 733/822 email had one
section which consisted of one block of metadata 
followed by one block of ASCII text. most emails
were "small" so the machinations required for manipulating
each of the two parts were relatively simple and
didn't seem that inconvient. It also made interfacing
with v6-era Unix text commands trivial.

NOW - email is a sequence of metadata/data structures
which must be processed independently in vastly different
ways for almost all purposes. The semantic structure is
independent of how email is stored. If it's linearized
into a single file, like MH works now, then there are 
interleaved blocks of metadata and data. or it could 
use a directory with each of the metadata/data sections
linearized into distinct files. or the metadata and
data with sections could be split as well, making
it easy to get to the components of the sections
as easily as the individual sections. Of course
this makes an individual message a "directory",
or rather encourages thinking about a mailstore
using directories as an implementation. Judicious
naming of sections and their metadata/data parts
could still make Unix interfacing relatively
simple for very simple cases. 

But lets look at an analogy.

Once upon a time, the "ps" command worked by
snuffling around in /dev/kmem and /dev/mem
with intimate knowledge of kernel data structures.
This meant that any change to the proc structure
required recompiling at least "ps" and probably
a few more things. It was also a security nightmare.

It was finally realized (and demonstrated in Plan 9)
that the information about a process was quite
independent of how and where that information is 
stored in the bowels of the system. Hence a "filesystem"
interface was created that externalized that information
in an easy-to-use form. Now "ps" is not much more
than an awk script that doesn't have to get whanged
every time something happens under the covers.
it also enjoys the same protection machinery
that governs access to other files. And the
image of a process is a directory full of little
files that appear to contain text strings.

Reasoning by analogy, I see no reason why a
mailstore which provides a filesystem interface
which makes messages appear as directories
containing well-named files linearizing
each of the subdivisions of the message
such that the metadata and data of each
section can be readily paired-up while
each can also be processed independently.

Note that what I just described is the 
mailstore semantics *as seen by MH commands*.
I make this distinction because MH commands
are willing to use specific functions to 
read and write the mailstore while other
external (non-MH) commands are not written
that way.

So how to export the semantics to external
commands?

one way is to implement the mailstore as
message directories containing all the
piece-parts in files just as the abstraction
describes. we might need an "mhname" command
to avoid disclosing naming conventions, or
just expose those. maybe that's easier.

another alternative is to provide an interface
to the mailstore via 9P over a reliable byte stream.
for systems that have 9P filessystem client code,
that's an easy way to get the mailstore interface
abstraction regardless of the implementation.

if there is no 9P filesystem client code for
a target platform, yet another egregious but
immensely useful hack is possible: a dynamic
libray wrapper. Run an external command in
a subshell with alters the dynamic linking
sequence to interpose a "shim" that captures
filesystem system calls and then either 
forwards them to the normal syscall interface
or handles mailstore operations itself.
That handler could operate either locally
or over 9P or both. this might introduce
some additional file syntax for external
commands like 
         grep -i bozofest mh:*/header1

(the * will see all the msg directories in
the current folder so grep will do the
right thing.


My point is that the implementation of the
mailstore and the semantic model of the mail store
have been intertwined since the beginning of MH-Time.
That kind of coupling has broken down time and again
in Unix history, but in most cases, we've been able
to sort out how to keep most of the good stuff
without too much inconvenience.

Don't shoot, Robert, but
I think it's time MH took its turn in the refactory.

  -mo