Re: Patching Octave-MPI

[Andy Jacobson]

>>>>> "JWE" == John W Eaton <address@hidden> writes:


    JWE> There is also a statement in the MPI docs that I saw saying
    JWE> something about being sure to initialize MPI before doing
    JWE> much of anything (I think I/O and opening files were
    JWE> mentioned).  Does anyone know the rationale for that?  What
    JWE> difference would that make?  If it really does make a
    JWE> difference, then there is another reason to put the call to
    JWE> MPI_Init in the core Octave.

I believe that all I/O is implementation-specific.  It is omitted from
the MPI-1 standard
(http://www.mpi-forum.org/docs/mpi-11-html/node6.html#Node6).

I'm only familiar with the LAM implementation of MPI, and in that case
one I/O issue involves determining whether or not stdout and stderr
are (a) linked to a terminal, (b) need to be forwarded to a remote
terminal, or (c) dumped to /dev/null.  It's my understanding that these
decisions are implementation-dependent, so I expect that MPICH handles
them a different way.

    JWE> This is what I was thinking also, that we would

    JWE>   * Compile support for MPI into Octave based on configure
    JWE> flags (so you could omit it completely and still build a
    JWE> working Octave.

    JWE>   * Even for Octave built with MPI support, only enable MPI
    JWE> functionality at run time if Octave is given the appropriate
    JWE> command-line option.

This seems like a perfectly reasonable solution to me.  However, I
would like to point out a couple of human interface issues.

MPI apps are usually started by some external code.  In LAM's case you
first have to start the LAM daemon on all the computers (via the
"lamboot" program).  Then you call "mpirun", which takes
responsibility for executing the member programs which make up the MPI
application (e.g. loading N instances of octave on M computers).
After the application is finished, you generally have to shut down the
LAM daemons manually (via "wipe" or "lamhalt").  Users will have to
know the details of starting an MPI application according to their MPI
implementation (LAM, MPICH, etc) in order to get octave to run in
parallel.

In addition, users need to know the compilation and linker flags
necessary to get an MPI-aware octave to compile successfully.  Once
more, this is implementation-specific.  I don't know whether autoconf
can be made smart enough to figure out how to do this.  LAM very
unfortunately hides these flags from the user by wrapping the compiler
in a script.  While it is possible to determine the proper flags from
this wrapper script, this scheme feels like a needless obfuscation to
me.

Finally, for developers, debugging parallel applications can be quite
challenging.  Enough said, I hope.

Empirically, I have found that these are fairly high barriers to the
use of MPI codes by non alpha geeks.  I think MPI-enabled octave,
especially if linked to ScaLAPACK or some such, would be very useful
to some people, and raise a lot of eyebrows--but it is unlikely to
ever be easy to use.  

I suggested one particular balance between access to low-level
functionality and high-level convenience with the MPI patches to
octave-2.1.31.  I thought the best part was that you could send
arbitrary octave_value objects between nodes (via a minor hack to
load/save).  Alex Verstak struck a different balance by providing a
raw interface to the MPI send/receive functions.  This is very
appealing just as a means to learn how parallel applications work.  I
think it would be great if both levels were available in the final
product.

        Regards,
                Andy

-- 
Andy Jacobson

address@hidden

Program in Atmospheric and Oceanic Sciences
Sayre Hall, Forrestal Campus
Princeton University
PO Box CN710 Princeton, NJ 08544-0710 USA

Tel: 609/258-5260  Fax: 609/258-2850