Re: Using OpenMP in Octave

[Jaroslav Hajek]

On Mon, Mar 29, 2010 at 6:29 PM, Søren Hauberg <address@hidden> wrote:
> man, 29 03 2010 kl. 11:50 +0200, skrev Jaroslav Hajek:
>> Unfortunately, it confirms what I anticipated: the elementary
>> operations scale poorly. Memory bandwidth is probably the real limit
>> here. The mappers involve more work per cycle and hence scale much
>> better.
>
> Bummer :-(  Not all that surprising, but still...
>
>> I know Matlab does
>> it, but I think it's just fancy stuff, to convince customers that new
>> versions add significant value.
>
> The sad part is that it actually seems to work. I've heard several
> people praising the parallel stuff Matlab does even if they haven't
> actually benchmarked it.
>

I wouldn't exactly call it sad. I'm sure there's a lot of value in
Matlab's mt support. But the marketing is clever and pretends that
there's even more. I recall seeing a "what's new" page for some
version 1-2 years ago, where they mentioned multithreaded prod, sum,
plus, minus etc. Since I already had some experience with
multithreading and OpenMP, I thought it was a bit funny; just like
OpenMPing xAXPY and xDOT and call the result "multithreaded BLAS". It
sounded convincing, because the list included a lot of functions that
are often used - but I got a vague impression that the whole stuff is
meant for quiche eaters :) But of course MathWorks doesn't want to
miss even quiche eaters with Matlab, so why not. Marketing is done by
all companies, not just MathWorks.


>> One area where multithreading really helps is the complicated mappers,
>> as shown by the second part of the benchmark.
>> Still, I think we should carefully consider how to best provide parallelism.
>> For instance, I would be happy with explicit parallelism, something
>> like pararrayfun from the OctaveForge package, so that I could write:
>>
>> pararrayfun (3, @erf, x, "ChunksPerProc", 100); # parallelize on 3
>> threads, splitting the array to 300 chunks.
>>
>> Note that if I was about to parallelize a larger section of code that
>> uses erf, I could do
>>
>> erf = @(x) pararrayfun (3, @erf, x, "ChunksPerProc", 100); # use
>> parallel erf for the rest of the code
>
> This approach could work. One potential annoyance is that you would have
> to do this in every function that uses 'erf' as the function handle is a
> local variable. In some situations this type of fine-grained control is
> great, but often you just want to be able to say, "parallellise stuff
> for me". This is why OpenMP is so successful even though many experience
> sub-optimal performance.
>

But OpenMP isn't about "just parallelize stuff for me". With OpenMP
you must explicitly pick loops you want parallelized; and you may need
to provide some guarantees to the compiler, tune scheduling method,
insert barriers etc.
It is way more complicated than auto-parallelizing, but can deliver
much better results, while still avoiding the parallel-from-scratch
code structure needed for MPI. *that* is why OpenMP is so successful.

>> If we really insisted that the builtin functions must support
>> parallelism, I say it must fulfill at least the following:
>>
>> 1. an easy way of temporarily disabling it must exist (for high-level
>> parallel constructs like parcellfun, it should be done automatically)
>> 2. the tuning constants should be customizable.
>
> I agree with this. It should be said that OpenMP allows you to control
> some stuff (like number of used cores) at run-time.
>


Yes. But I think we can't just wrap omp_set_num_threads and
omp_get_max_threads, because the value is global and could possibly be
changed by external libraries, such as BLAS.
I think that we need to keep our own variable for the max num of
cores, and always set it just prior to the operation.

-- 
RNDr. Jaroslav Hajek, PhD
computing expert & GNU Octave developer
Aeronautical Research and Test Institute (VZLU)
Prague, Czech Republic
url: www.highegg.matfyz.cz