Re: Shared memory interface to Octave

The only reason I chose shared memory instead of named pipes(FIFOs) is because 
of the speed issue.  I was researching on the web for information on how long 
it takes to transfer data.  As I had read through the mailing lists, it seems 
that some of the GUIs that have been started in the past used sockets to 
transfer the data between the GUI and the core of Octave, but that the speed 
was a limiting factor on its usefulness.  I was trying to find the fastest 
possible method.  The following url gave a rundown on the times for different 
types of inter-process communications:
http://opensource.or.id/~dewa/References/Unix%20IPC/node10.html
It indicates that transfering 256KB via named pipes would take ~32 ms while 
only ~256 microseconds with the shared memory.
It should be noted that the document is somewhat old and was tested on Linux 
kernel 1.2.9 on a 90 MHz machine.  It would be expected to be much, much faster 
on modern day 1GHz and above machines.  Named pipes may be more optimized in 
more recent versions on Linux (and other *nix-like OSs).  
I agree with you that the named pipes would be much easier.  Is would be very 
easy to redirect stdin and stdout from and to the pipes.  I could then open two 
other pipes for used as a binary transfer channel.  As an aside, I have yet to 
figure out how to redirect stdout to a block of shared memory other than 
redirecting it to a file, reading it back in, and then writing it to the shared 
memory block.  This process may take longer than just using named pipes.

Named Pipes Method
1) Redirect stdout and stdin to read from and write directly to the pipes 
between the GUI and the octave core.  This should provide a pretty transparent 
interface to octave and could be implemented in any language.  I will need to 
prototype this up, but from what I can tell, octave shouldn't know the 
difference between receiving the input from its regular console or the 
redirected stdin.  Same for stdout.
2) Create two other named pipes for communicating control and variable 
information between the GUI and the octave core.  For example, the GUI would 
listen for a ctrl-c and send it via this second pipe in order to interrupt the 
execution of the current command.  Also, after the execution of each command, 
the octave core would send a list of all the variables and their types and 
sizes back to the GUI via one of these pipes.

Thanks for the suggestion of named pipes.  I am going to do my own performance 
tests, but the advantages look to outweigh the speed hit, so I will most likely 
end up using them.

John Swensen


Quoting Russell Standish <address@hidden>:

> I don't see why you need to use shared memory? Surely opening a pipe
> connection between the Octave's standard input and your control
> stream, and another one to its standard output suffices to communicate
> between you GUI process and the octave process.
> 
> The pipe construct can be easily converted to a distributed memory
> model by means of ssh or similar ilk. Pipes are also far easier to
> program than shared memory segments.
> 
>                                       Cheers
> 
> address@hidden wrote:
> > 
> > 
> > Since I am new to the maintainer list, hello to all before I get into my
> questions.
> > 
> > I have used Octave off and on through college (intermixed with Matlab).  I
> have
> > always thought it would be good to have a GUI interface and have read a
> some of
> > the mailinglist archives about peoples' suggestions on how to do it. 
> After
> > reading through all of these, and drawing on a little past experience, I
> think
> > that using the shared memory functions shmget, shmat, shmdt, and shmctl
> would be
> > an ideal way of implementing the communications between the GUI and the
> octave core.
> > 
> > Because of this, I am asking for a quick rundown on how console I/O is
> > implemented in Octave.  I could wade through the sources, but I feel it
> would be
> > beneficial if I could get a 1 page summary from someone who has done the
> > development.  This will make my wading that much more productive.
> > 
> > Back to the topic of what I am trying to do, I used the octave-parallel
> socket
> > interface from octave-forge as a model and implemented a pretty
> rudimentary
> > passing of commands and results via shared memory.  Many commands work,
> but
> > other crap out on me.  My sources are broken right now, so once I get them
> > working again, I will probably post some of the erro output for help on
> > determining where the problem is.
> > Another benefit of using the shared memory is that on clusters that use a
> shared
> > memory model, as far as I can tell, this method should work as a means of
> using
> > Octave in a distributed manner an be much faster than using sockets.
> > 
> > I would like to know other poeples suggestions, comments, and criticisms on
> my
> > plan.  I have used shared memory some in a few application I have written
> > before, but am by no means and expert.  
> > 
> > John Swensen
> > address@hidden
> > 
> > -------------------------------------------------
> > This mail sent through IMP: http://horde.org/imp/
> > 
> 
> 
> 
> ----------------------------------------------------------------------------
> A/Prof Russell Standish                Director
> High Performance Computing Support Unit, Phone 9385 6967, 8308 3119 (mobile)
> UNSW SYDNEY 2052                       Fax   9385 6965, 0425 253119 (")
> Australia                              address@hidden             
> Room 2075, Red Centre                    http://parallel.hpc.unsw.edu.au/rks
>             International prefix  +612, Interstate prefix 02
> ----------------------------------------------------------------------------
> 




-------------------------------------------------
This mail sent through IMP: http://horde.org/imp/