Re: Where to look next for performance improvements

[Rik]

On 06/25/2014 12:55 AM, address@hidden wrote:
> Message: 4
> Date: Wed, 25 Jun 2014 08:35:35 +0200
> From: Julien Bect <address@hidden>
> To: Rik <address@hidden>, Octave Maintainers
>       <address@hidden>
> Subject: Re: For loop benchmarks between Octave versions
> Message-ID: <address@hidden>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> Le 17/06/2014 02:39, Rik a ?crit :
>> > I find that this changeset:
>> >
>> > 12920:5d18231eee00 Extend profiling support to operators.
>> >
>> > produces a 50% slow down when using the for loop benchmark. Unless we 
>> > can figure out a lower footprint way to do profiling we might be stuck 
>> > with this part of the slowdown. There is still another 50% which seems 
>> > to be due to other accumulated cruft, i.e., I didn't find a smoking 
>> > gun changeset.
> I have built revision 12922 (Thu, 04 Aug 2011, default branch, just 
> after the above-mentioned changeset)with my "inlining +template" patch 
> for the profiler.
>
> My benchmarks indicate that this patched revision is even slightly 
> better than Octave 3.4.3 :
>
>                               Octave 3.4.3 r12922 + patch
>           doubleForLoop/1     114.9 [  0.9]     108.4 [ 1.9]   evol:  -5.6%
>         doubleForLoop/a=1     165.5 [  0.6]     154.5 [ 1.8]   evol:  -6.6%
>         doubleForLoop/a=b     190.2 [  0.7]     169.2 [ 0.9]   evol: -11.0%
>       doubleForLoop/a=a+b     171.5 [  1.2]     143.1 [ 1.1]   evol: -16.6%
> doubleForLoop/a=sin(b*i)     224.6 [  1.6]     207.1 [ 2.9]   evol:  -7.8%
>           vectorOfSquares     326.8 [  4.0]     336.5 [ 9.4]   evol:  +2.9%
>                    mandel     126.4 [  1.6]     120.7 [ 1.1]   evol:  -4.6%
>                       fib     336.2 [  1.1]     335.8 [ 1.1]   evol:  -0.1%
>                 parse_int    1414.1 [ 16.0]    1377.1 [ 24.2]   evol:  -2.6%
>                 quicksort     632.8 [ 11.3]     622.7 [ 10.9]   evol:  -1.6%
>                    pi_sum    8678.9 [ 84.2]    7452.6 [ 27.6]   evol: -14.1%
>             rand_mat_stat     315.9 [ 10.8]     309.1 [ 8.5]   evol:  -2.1%
>              rand_mat_mul     387.3 [ 20.1]     363.2 [ 20.1]   evol:  -6.2%
>                   printfd    2404.7 [ 30.9]    2310.7 [ 5.7]   evol:  -3.9%
>
>
> This suggests that the additional slowdown between those revision and 
> now has nothing to do with the profiler (?).
>
> Any idea about where to look next ?
6/25/14

Julien,

Unfortunately, I think it gets a lot harder from here.

I saw roughly a doubling in execution time from 3.4 to 3.6 and another
doubling from 3.8 to 4.0.  On my computer, the option
--enable-atomic-refcount, is entirely responsible for the doubling I see
between 3.8 and 4.0.  The profiling changeset that I found with bisection
was responsible for 50% of the slowdown between 3.6 and 3.8.    Together,
these two changes account for 75% of the slowdown.   If we believe the
Pareto Principle then 80% of the effect is due to just 20% of the causes
and we've already found those causes.  The remaining 25% is going to come
from little improvements here and there spread out over many changesets. 
This was certainly what I found while doing the bisection.  From changeset
12922 the performance slowly degraded, but I didn't find anything that
accounted for a large chunk of the slowdown.

Things get lost on the maintainer's list.  Could you submit your profiling
changeset to the patch tracker on Savannah?

Thanks,
Rik