Re: [Gnu-arch-users] Thoughts on VU

[Tom Lord]

    > From: Matthew Dempsky <address@hidden>

    > vu sets the stage for some really neat and powerful ideas, but I think
    > we need a mechanism so that sub-processes can inherit the same handler
    > stack or something.

Yes please, but go further.

I have incomplete but vivid thoughts on the matter.

The VU handler stack defines a complete virtual filesystem composed
out of more primitive file systems according to some rule.  

One idea is to make the set of rules for defining stacks a universal
set.   In other words, if my process is using some stack I can tell
your process the rule that defines that stack, and then your process
can run the same stack.  (Maybe my stack rule points to some local
directories that your process doesn't have access to --- that's fine,
it should just be an error if your process tries to access those
dirs.)

So we could have an exchange format for these rule sets.   Process
could trade them.   You wouldn't be limited to just inheritence by 
subprocesses (although that could be part of it).

I think that stack rules have a kind of algebra to them.   I can take
my current rule and add your rule to it, perhaps overriding mine or
perhaps being overridden by mine.   It'd be interesting to implement 
that algebra.



    > For example, if I write hash (Hackerlab shell) and it has code so I
    > can cd to /.../ssh/address@hidden/home/jivera, it will kinda suck
    > when I try to run any utilities on files in the directory because when
    > the shell forks and calls exec, the vu stack is reinitialized --
    > without the ssh vu vtable -- and fails.  (I think the issue of
    > non-vu-enabled tools failing is most likely unavoidable, but extra
    > points to anyone who can solve it.)


Bingo.


    > Since the code to implement all of the vu stack needs to be present in
    > both processes, all the handlers need to be in either hackerlab or in
    > shared libraries distributed with applications that are installed
    > somewhere and automatically loaded by hackerlab.  This seems like it
    > would add a lot of (potentially useless) complexity to each
    > processes's startup.

I wouldn't mind seeing a small set of built-in rules, plus a larger
set of dynamically loadable rules -- implemented by a mix of dynamic
loading of libraries and dynamic loading of extension language code.
I think that (the soon to be announced "furth" extension language)
will be compact enough to embed in nearly all programs.h

Dynamically loaded things could be named by URLs.   People should be
able to publish new filesystem extensions with ease comparable to how
they currently publish web sites.


    > If we don't want that solution, we'll need either OS support (not
    > really portable) or an IPC-based vu handler that'll connect to *some*
    > process -- probably the parent process which would then need to add in
    > all the hair of handling asynchronous requests from children for
    > filesystem operations (and might even then need to pass requests to
    > its parent).  Yick.

That's an interesting long term question.   Suppose I send you some VU 
stack rules.   Do they have to be translated?   E.g., my rule mentions
some local filesystem.   You don't have that but, it turns out, the
same filesystem is available via NFS.   So where and how does my rule
get translated so that the local references are replaced by NFS references?


    > Alternatively, we could write a vud (virtual unix daemon) which is
    > responsible for implementing a lot of vu requests and then have an
    > IPC-based vu handler that dispatches to it.  

"Alternatively?"   I think the word you want is "Also".   It would
solve different problems and would be useful, I think.

    > This would mean we only need to worry about hairy asynchronous
    > requests in one process.  We can probably also use some
    > environmental variables for specifying how to connect to the
    > daemon and what data to send to it.  Things like ssh connections
    > could then be tracked among children processes and when I cd to
    > /.../ssh/address@hidden/ in hash, I can still run all my
    > utilities without needing to enter my password again.  (Of
    > course, this opens a few security concerns that need to be
    > carefully addressed, but I expect nothing that any
    > microkernel-based OS's filesystem needs to handle.)

    > I imagine a more likely situation, however, would be to have multiple
    > vuds capable of different magic.  Then, the regexp handling in vu
    > would be used for deciding which vud to connect to and communicate
    > with.

    > How far off am I?

Not.

-t