Re: Question on performance, coding style and competitive software

[Jaroslav Hajek]

On Tue, Apr 21, 2009 at 6:12 PM, Alois Schlögl <address@hidden> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
> As some of you might know, my other pet project besides Octave, is
> BioSig http://biosig.sf.net. BioSig is designed in such a way that it
> can be used with both, Matlab and Octave. Mostly for performance reason,
> we cannot abandon support for Matlab [1,2]. Octave is a viable
> alternative in case the computational performance is not important. In
> order to decide on the future strategy of BioSig, I hope to get answers
> on the following questions:
>
> 1) Core development of Octave:
> At the meeting of Octave developer in 2006, the issue was raised
> that the Octave is about 4 to 5 times slower than Matlab [1]. (I
> repeated the tests just recently, the results are attached below, and
> show a difference of factors up to 13, average ~5) This issue is most
> relevant for large computational jobs, were it makes a difference
> whether a specific task takes 1 day or 5 days. Is anyone working to
> address this problem?

Well, I hear about your problem for the first time, so it's hard to
imagine myself
being working at it, and I think the issue is similar with everyone else :)
OK, enough of being ironic. There has been a number of performance
improvements in Octave in the last year, and more are coming. Octave
is already faster than Matlab in a number of areas, while being slower
in others.
The most striking example is JIT compiling of simple loops in Matlab -
Octave lacks a similar technique. This project is estimated to by a
highly complex task and probably won't happen without some specific
funding.
It can be circumvented by writing compiled code, which is probably
what most packages do.

> Is there any hope that the performance penalty
> becomes smaller or will go away within a reasonable amount of time ?

Maybe. Let's see what you can do:

1. contribute performance improvements
2. support someone else to do it
2a. financially
2b. by providing benchmarks (as simple as possible), test cases, suggestions...
2c. moral support

of course these options differ in their effectiveness. Speaking as
probably the most active performance zealot amongst Octave developers,
I daresay that the cost of a single Matlab license (~2400 EUR) would
be more than a pleasant compensation for all the performance
improvements I worked on last year, and these make Octave outperform
Matlab in certain areas (array indexing & permuting, diagonal and
permutation matrices).

I can look at some of your benchmarks if it's not too complicated.

> 2) Coding style:
> Octave understands a superset of commands compared to matlab, and it
> seems the current policy is to enforce the "octave style" and make the
> use of toolboxes incompatible for a use with Matlab.

"Enforce" is only true for code going into Octave. Otherwise, you're
free to use whatever coding style you wish. There are packages even on
OctaveForge that are Matlab compatible.
There are purposes for using Octave-specific features (code clarity,
robustness, efficiency). It depends on the developer whether the gains
outweigh the incompatibility with Matlab.

> Is not it sensible to write platform-neutral applications ? Specifically, is 
> not it in our
> own interest (wider usage make the code more robust) that matlab users
> are not "forced" to buy additional toolboxes but can use open source
> toolboxes e.g. from octave-forge?
>

Again, it depends on your intent. With most of my packages, I don't
care whether they run on Matlab. Nobody has even asked for it.

>
> 3) Scope of Octave and Octave-Forge:
> Open source software has its own merit, but sometimes also other factors
> (e.g. additional costs in hardware, energy supply and cooling systems,
> energy efficiency = "green computing") need to be considered. Given the
> fact that octave-core is currently slower for some tasks, it is worth
> considering to use proprietary mat-engine. The question is whether
> Octave and Octave-forge should provide support of toolboxes for matlab
> users too, or whether these users should go somewhere else? What do you
> think ?

I think Octave's and OctaveForge's aims are quite clear. I've seen no
opposition against Matlab-compatible packages on OctaveForge.
Octave-only packages are OK, too. (Maybe that's what you're opposing?)
If you're thinking about Matlab-only packages, the maybe that's OK
too, but I don't think OctaveForge is a good place for those.

regards

-- 
RNDr. Jaroslav Hajek
computing expert & GNU Octave developer
Aeronautical Research and Test Institute (VZLU)
Prague, Czech Republic
url: www.highegg.matfyz.cz