octave-mpi questions

[John W. Eaton]

On 13-Feb-2003, John W. Eaton <address@hidden> wrote:

| On  8-Feb-2003, Andy Jacobson <address@hidden> wrote:
| 
| |         I recently received a request from Martin Siegert to help get
| |         the MPI patches working with octave-2.1.44.  (There appear to
| |         be some issues with changes in the standard C++ library, but I
| |         haven't looked into the problem in much detail yet).  I
| |         vaguely recall that you and I recently had an email exchange
| |         in which you mentioned that you had looked into incorporating
| |         some MPI capability into the standard distribution.  Not
| |         wanting to duplicate efforts, I wonder whether you have
| |         undertaken such a thing, or if Martin and I should do so.
| 
| Sorry for the delay.  I stalled on this because after I looked at the
| code, I started thinking that it would be better to have thin wrapper
| for the MPI library, then implement higher level functions in
| M-files.  But that's going to be a lot of work, and I don't understand
| the MPI library as I don't use it (yet).  So I suppose it would be
| best do go ahead and include the functions that you have written with
| a note that what we probably really want is a more complete interface
| to MPI.  So, let me take another look at it.  Probably it will not be
| hard to get it in to the CVS archive quickly.

OK, I have adapted the patches for the current Octave sources.  I
haven't checked the changes in yet because I have a few questions.

It seems to me that it would be best for this code to be loaded
dynamically, and have the mpi library linked only to the dynamically
loaded .oct file so that users who don't want MPI don't have to pay
for it (at all) and so we don't have to have two different binaries.

I can see a couple of ways to do this.

One is to add a --mpi option for Octave which will cause Octave do do
the inits (but by using feval() to call .oct files so that the MPI
library doesn't have to be linked to the main Octave binary).  You
could run Octave with a command like

  mpirun OPTIONS octave --mpi

to start multiple Octave processes on one or more systems and give
yourself a prompt for the master process (or just run a script with
the master).  This seems fairly straightforward, but still requires
Octave to be modified to handle MPI.

Another way to add the MPI functions to Octave would be to not modify
Octave's startup procedure at all, but require the user to call some
kind of mpi_start function that does the initialization and also
includes the functionality of mpirun (if that is possible -- I suppose
we will need it to somehow tell the other processes that they should
perform the MPI startup steps too).  This startup function can also
call atexit ("mpi_quit") (or similar) to ensure that MPI is stopped
properly when Octave exits.  The user can also call mpi_quit without
exiting the main Octave process, then when mpi_quit is called by
atexit it will do nothing.  The problems I see with this approach are
the warnings (which seem strange to me) about not doing much before
calling MPI_Init and passing pointers to the actual argv/argc from the
C main.  (Surely this is not a requirement, because there are a number
of add-on packages for Matlab that use MPI and are implemented entirly
in .m and mex files.)

I like the idea of having the flexibility of starting and stopping MPI
processing on the fly, but I don't know whether that is practical.

Comments?

Thanks

jwe