[Vrs-development] SEE and Goldwater integration (was Re: [DotGNU]Network SEE architecture, v2.0.1)

[Stephen Compall]

Note: I will be gone (that is, offline) from before 11 Oct 2200 UTC to 
15 Oct sometime probably.

First let me stress that I believe that my Network SEE design fits both 
"The Right Thing" and "New Jersey" approaches towards framework design. 
That is, both the interface and implementation are simple. This is 
important to me, if not in the greater scheme of things.

Anyway, here are my thoughts on these concepts.

Chris Smith wrote:
> You make much of pipes and other IPC ideas in your documents which
> the VRS also requires (IPC that is). The reason I got involved in
> dotGNU (apart from it being a damn important thing to be involved in)
>  was that I had a language independant middleware that could be used
> to tie all the process components of a the VRS together. In addition
> to this, it also provided the same IPC mechanisms transparently
> across networks. Suddenly a simple single machine application could
> become distributed without writing any network code! This was very
> attractive from the VRS standpoint.
>
> The bit when a SEE gets a request that IT can process is the same for
>  the VRS. Even the logical layering (right down to the fact that
> you've described them as processes [seedaemon, see*port etc]) is the
> same. We're using GW to tie these processes together. I wonder if the
> SEE can't as well.

I'm sure it can be done, the only question is: how light is the 
protocol? See comment 4

> I notice that you suggest using Serveez. I'm not familiar with that.

Serveez is a server framework that allows you to write callbacks that
act as servers whereas Serveez handles all the network communications.
It has some weaknesses, but was evidently designed to be compilable with
MSVC. That is valuable to me, because I don't know anything about
writing code that can compile with MSVC, and don't really want to learn.
Good for cannibalizing y'know :)

> VRS is being built around a middleware that does all the
> single-machine inter-process-comms for you and also inter-machine
> comms too. However, we (the VRS) will still need some sort of network
>  presence for the request/reply transport. How this will be done is
> undecided - possibly use off-the-shelf daemons like Apache and Jabber
> for this. What are your thoughts on this area for SEE?

see*user and see*port are the network players in the SEE model. SEE
clients and SEE daemons don't even know they're there; they only see the
FIFOs.

> So given that both our designs are so modular with a significant
> functionality cross-over, I'm kind of hoping that we can factor out
> the comminality (which is effectively the receive, decode, fetch,
> execute and return bit) and provide a SEEinterface module and a
> VRSinterface

My concern is not about the defined-ness of the API, but the weight. 
That is, I intend it to be trivial to write from-scratch 
see*port/see*user pairs, w/o using existing libs, provided that the 
language can open/close files and has an XML parser.

> One think that I need to keep in mind is that Goldwater Middleware
> has not been ported to Windows. Not a mammoth task, but does need
> inter-process shared memory. Mind you its API is very clean and
> straightforward, a Windows equivalent 'middleware' could be created
> that adhears to this API - sort of a GoldwaterLITE, but without all
> the performance benifits and scalability features. That way the
> application code stays the same, the architecure underneath changes
> instead. This might also be useful for un*x, for people who want to
> run a simple not-particularly-tunable VRS-nay-SEE-nay-VRS
> application.

Hate to say it, but the prospect of shared memory makes me a little 
nervous. :]

Anyway, cannibalizing Serveez network code and whatever else is there 
could be very useful toward that end.

> the white paper - just look at the picture (figure 1). And the
> incomplete Tutorial (yet another picture to look at). These pictures
> show the modular nature of Goldwater that we're using to tie the VRS
> components together. I imagine they look pretty similar to your
> diagrams of the SEE :o)

Yes they do...the ones in my head that is. You have seen the 
outside-my-head diagram :)

However, my head depicts startsee off to the side, then see*ports next 
to the network interface, and behind them various webservices, and all 
of the services being children of seedaemons (which, BTW, refers to 1 
process), but it's not behind the children because the see*ports need to 
talk to it too. Actually, I'm thinking of moving the FIFO creation to 
startsee, because seedaemons *must not crash ever*

see*users are not important in this diagram, they're just clients.

The message queues (by which you mean FIFOs, I'm sure) are not in a big 
block, they're really independent, only names are decided rather easily 
(index to number of connection see earlier post (and the xml block in 
see-arch 2.0.1). Timeouts are implemented through deleting the FIFO.

Hmm, I think startsee does the client-side FIFOs and seedaemons does the 
server-side FIFOs. Yep, that's the way to go.

Finally: it's not so much the VRS and SEE overlapping, it's SEE and 
Goldwater, I think. Make of this what you will.

--
Stephen Compall
Formerly known as S11001001
DotGNU `Contributor' -- http://dotgnu.org

To be a hacker, one had to accept the philosophy that writing a
software program was only the beginning. Improving a program was the
true test of a hacker's skills.
        -- Sam Williams, "Free as in Freedom"

According to Stallman, improving software programs was secondary to
building them in the first place.
        -- Sam Williams, "Free as in Freedom"