Re: Multithreaded FFTW3 in octave?

[Mike Miller]

On Tue, Dec 18, 2012 at 11:51:00AM +0100, Andreas Weber wrote:
> On 18.12.2012 05:08, Mike Miller wrote:
> >The configure changes don't look quite right to me. I wouldn't make
> >the HAVE_FFTW3_THREADS condition dependent on a separate fftw3_threads
> >option. The way you have it here I don't think I can configure
> >"--with-fftw3=somefftwlib" and have it detect whether some
> >FFTW-compatible library has the threading API or not.
> Is there any "FFTW-compatible library"? I don't know one and as I
> kan see this also wasn't possible before my changes.

IMKL, for example. My point is, I can tell Octave's configure
--with-fftw3="-lxyz -labc -lm" and it will take my word for it and test
that set of libraries to see if the FFTW interface is present. Same with
any of the other libraries used by Octave. By extension there should be
a way to test that set of libs to see if they support the FFTW threading
functions.

> And you do not want to be able to disable multithreading with
> "--disable-fftw-threads?
> So just check if its possible to compile and link with multithreaded
> libs, if not try the single threaded,
> if that fails try the FFTPACK?

I like --disable-fftw-threads, see below.

> >It's also
> >linking with both fftw3_threads and fftw3 (non-threaded).
> From the FFTW3 manual:
> ...programs using the parallel complex transforms should be linked
> with |-lfftw3_threads -lfftw3 -lm| on Unix...
> also  nm -D --defined-only libfftw3_threads.so doesn't show any DFT
> function.
> Please correct me if I'm wrong but I think ist correct to link with both.

You're right, I was getting that implicitly with my tests because it's
pulled in by the fftw3_threads shared library even if you don't link it.
But you do have them backwards in FFTW_XLIBS.

> >  I'd rather
> >see it test the existing FFTW3_LIBS and FFTW3F_LIBS whether they
> >implement one of the threading functions.
> See above
> >I'm not sure off the top of my head what's the cleanest way to do it
> >that way *and* make it automatically use the libfftw3_threads library
> >if it's installed. And how to express whether to use libfftw3_threads
> >or libfftw3_omp? Worst case, we could require users to configure with
> >"--with-fftw3=fftw3_threads --with-fftw3f=fftw3f_threads", this has
> >pros and cons. I think I prefer that for now, it's the easiest and
> >safest and we can always change it later.
> I can't see any benefit from doing this and want to ask some
> autotools pro for that.

Now that you've corrected me about the relationship between the libs,
how about something along these lines (pseudo-autoconf-shell), I think
it does what you intended and addresses my concerns:

OCTAVE_CHECK_LIB(fftw3, ...) as it currently is

AC_ARG_ENABLE(fftw-threads, ...) where the choices are "no", "yes",
"threads", or "omp". The "yes" choice would be equal to "threads" and
would be the default. This one enable option would apply to both fftw3
and fftw3f.

if -n "$FFTW3_LIBS" ...
    if $enable_fftw_threads is not "no"
        first try FFTW3_LIBS by itself to see if it has thread function
            if it works, no need to change FFTW3_LIBS
        then try prepending -lfftw3_${enable_fftw_threads} and try again
            if it works, prepend to FFTW3_LIBS
        if either test works, define HAVE_FFTW3_THREADS
    fi
fi

And repeat for fftw3f. I could parameterize this into an autoconf macro
that can simply be called twice for fftw3 and fftw3f. No need for the
extra set of FFTW3_THREADS_LIBS variables.

> >I would set the default number of threads to 1 at startup. Matches the
> >default behavior of both the non-threaded and threaded FFTW libraries.
> Okay, I thought a dual core is standard nowadays and users would
> benefit from this even when they didn't notice the change that they
> now can set the number of threads with fftw("threads",...)
> 
> Another way could be to set the default number of threads to the
> number of available cores on startup but,
> 1.) I don't like such type of "hidden automation"
> 2.) We first would have to implement some platform independent way
> to get the number of available CPU cores

Agreed about the automation, I don't want that. Yes, multi-core is
becoming more common, but you don't know if someone wants ffts using up
all of it by default. The only default that makes sense to me is 1.
That's the way FFTW behaves if you don't call plan_with_nthreads.

That's my case. If you still think the default should be 2, that's fine.

> >There is some trailing whitespace on a few lines that should be removed.
> Okay, done that
> >I already fixed the method / do_method bug recently on the stable
> >branch (http://hg.savannah.gnu.org/hgweb/octave/rev/f63a4f23bfe7), I
> >think it simply hasn't been merged into the dev branch yet, so don't
> >include that in this patch.
> Ah I didn't notice that you've fixed that on stable. I stumbled over
> this problem some days ago and had to fix it for my tests with fftw.
> When do you merge this from stable into default?

Jordi can answer that and take care of merging it.

> >Other than that, looks real good.
> Thank you very much. I want to talk with Jordi on IRC gerading some
> points and submit then a new patch.
> And I appreciate any further comments.
> 
> I've further made changes that the "threads" getter and setter is
> only compiled if defined (HAVE_FFTW3F_THREADS)
> and the plan only gets discarded when there is a change in the
> number of threads.

Why not keep the user-visible portion consistent and report an error
similar to how you had it? This makes sense to me:

octave> fftw ("threads")
ans = 1
octave> fftw ("threads", 4)
error: fftw: not available without multi-threaded FFTW

I got the "not available" text from other functions throughout Octave
with features that may or may not be compiled in but are always
callable.

-- 
mike