Re: OpenBLAS and performance

[Dave Love]

Pjotr Prins <address@hidden> writes:

> The last weeks I have been toying with OpenBlas and tweaking it boosts
> performance magnificently over the standard install we do now.

How so?  I haven't measured it from Guix, but I have with Fedora
packages, and OB is basically equivalent to MKL in the normal
configuration for AVX < 512.

> A configuration for Haswell looks like:
>
>   
> https://gitlab.com/genenetwork/guix-bioinformatics/blob/master/gn/packages/gemma.scm#L64

Why make it Haswell-specific?  The cpuid dispatch is the main reason to
use OB over at least BLIS currently.

> It will benefit python-numpy and R users greatly to use
> multi-threading (particularly). How do we make a flavour that supports
> this.

[I assume/hope it's not intended to default to multithreading.]

Fedora sensibly builds separately-named libraries for different flavours
<https://apps.fedoraproject.org/packages/openblas/sources/>, but I'd
argue also for threaded versions being available with the generic soname
in librray sub-directories.  There's some discussion and measurements
(apologies if I've referenced it before) at
<https://loveshack.fedorapeople.org/blas-subversion.html> -- not that
measurements sway people who insist on Microsoft R ☹.  Fedora should
sort out support for optimal BLAS/LAPACK, but those sort of dynamic
loading tricks are important in HPC systems for various reasons, and
seem rather at odds with the Guix approach.  I should write something
about that sometime.

If you do provide some sort of threaded version for Python, then as far
as I remember it must use pthreads, not OpenMP, though you want the
OpenMP version for other purposes, and I hadn't realized there wasn't
one currently.  That's covered in some combination of the OB and Debian
issue trackers.  I don't know if the same applies to R in general.

> Or are channels going to solve this for us?
>
> Btw the latest stable release worked fine too:
>
>   
> https://gitlab.com/genenetwork/guix-bioinformatics/commit/474524a5a0d57744c1727442b33d8f2889eb0397
>
> Pj.

Beware that 0.2.20 has one or two significant problems that I don't
remember, but could check.