Re: Octave Interpreter

[Stefan Seefeld]

Hi Mathew,

[I'm moving this back to the list, I assume you just accidentally hit
'reply' instead 'reply-all']


On 2014-10-06 00:07, Bipin Mathew wrote:
> Hello Richard, Stefan, et. al. 
>
>     Just to get us thinking about what implementing a distributed
> computing engine for Octave would take, lets consider a simple example
> and discuss what might be required. Just to get it rolling, lets look at 
>
> A = fft(X)
>
> where X is an instance of a dmat class (i.e. a "distributed matrix" ),
> in the sense that its contents are distributed across several servers.
> For the moment, i suppose we can consider fft to be an overloaded
> function specific to type dmat. Moreover, just for concreteness and
> simplicity lets think of X as a 2d matrix of size 200 rows by 10 columns.

OK.

>
> For a distributed implementation of fft, I reckon the first thing the
> top level fft function would need to know is "Where is X?". This could
> be gotten from a distributed data store of some kind, but suppose in
> the end we get something like this.
>
> X = { num_dimensions = 2
>     [1,100]  
>  ,[1,10],{server1/path_a1;server2/path_b1;...serverN/path_z1}
>     [101,200],[1,10],{server1/path_a2;server2/path_b2;serverN/path_z2}
>     [1,100]    ,[11,20],{server1/path_a3;server2/path_b3;serverN/path_z3}
>     [101,200] ,[11,20],{server1/path_a4;server2/path_b4;serverN/path_z4}
> }

OK.

>
> The first row for example says, all the elements contained withing the
> 1st -> 100th elements of the first dimension and the 1->10th elements
> of the second dimension can be found on
> servers/path  {server1/path_a1;server2/path_b1;...serverN/path_z1}
> (NFS? ). I also imagine instead of NFS paths, we can have identifiers
> to locations in memory of a slave process spawned on these servers.

Yes. I really think we should use standard APIs for this, rather than
inventing our own (API, transport protocol, etc.).
For example, it seems the existing MPI bindings I was just pointed to
would be a great starting point.
(Note that the pMatlab package itself also is layered over MPI Matlab
bindings.)

>
> The top level fft function would then spawn slave octave processes on
> the corresponding servers ( using a job handler ? ). The slaves would
> then load from disk and read into memory the the file they were
> assigned ( the file would be local to that slave. We should move
> computation not data ) and do a localized computation. In this case
> each slave does an FFT along each of the columns that are local to it
> and maintains that "sub"-fft in its memory.

Where these additional processes are spawned is an interesting question.
I doubt that an operation such as fft() would be the right place to do
it, though. For a proof-of-concept, I think we may as well consider the
MPI bindings, and just spawn octave itself via mpirun. Once this
mechanism is working, we could think of ways to delay the sub-process
spawning, to let users only run the normal single-node octave (acting as
a "controller"), and then spawn "engine" processes later from a new
"parallel_init()" function.

>
> The top-level fft function would then ask each slave process for its
> "sub"-fft and do the necessary computation locally to get the FFT of
> the complete vector OR, push the necessary multiplier matrix down to
> the slaves and have the slave multiply the sub-fft matrix by the
> multiplier matrix provided by the top-level fft function and viz-a-vie
> have a distributed matrix that represents the answer to the original FFT.

Yes, though, I think reinventing such an API is not a good idea. Rather,
we should use existing know-how, such as MPI.
In particular, the pMatlab package
(http://www.ll.mit.edu/mission/cybersec/softwaretools/pmatlab/pmatlab.html)
is layered over a Matlab MPI API
(http://www.ll.mit.edu/mission/cybersec/softwaretools/matlabmpi/matlabmpi.html),
so if we can manage to support an MPI API similar to that with the
Octave MPI package, we may get pMatlab support for free.

In other words, I see two main areas that need work:

* Assess the state of the Octave MPI bindings (and adding any additional
functionality required that's not yet covered)

* Consider ways to spawn sub-processes and set up MPI communicators
interactively


    Stefan

-- 

      ...ich hab' noch einen Koffer in Berlin...