Re: For loop benchmarks between Octave versions

[Daniel J Sebald]

On 06/18/2014 01:08 PM, Daniel J Sebald wrote:

> Comments:
>
> 1) Yes, something happened along the way to slow down looping or (more
> likely) variable (re)assignment. However, it looks like a gradual thing
> up until the GUI release and then a big jump associated with the GUI
> release, but I wouldn't conclude that the slow down is attributed to the
> GUI release given the sporadic sampling of versions.
>
> 2) I'd say look as far back as 3.5/3.6 because there was a very good
> improvement in efficiency with version 3.4.3 that was lost already with
> 3.6.4.

Here are some more results for slight variations on what is in the 
for-loop to see if we can discern exactly what is changing between versions:

--- Test Code ---
a = 1; b = 1; tic; for i=1:1000; for j=1:1000; a; end; end; toc
--- End Test Code ---

As non-root:

Version                                               Average
3.2.4                0.425301  0.421789  0.420683  |  0.42259
3.4.3                0.286903  0.292417  0.293619  |  0.29098
3.6.4                0.322811  0.321833  0.324628  |  0.32309
3.8.0                0.38697   0.401553  0.386148  |  0.39156
3.8.1                0.42775   0.425708  0.4329    |  0.42879
gui-release          1.05737   1.06017   1.05398   |  1.0572
  (no-gui-libs)      0.929348  0.922215  0.923098  |  0.92489
  (no-gui)           1.08166   1.08096   1.08397   |  1.0822
dev                  1.10937   1.10638   1.1032    |  1.1063
  (no-gui-libs)***
  (no-gui)           1.12748   1.12459   1.13151   |  1.1279


--- Test Code ---
a = 1; b = 1; tic; for i=1:1000; for j=1:1000; 1; end; end; toc
--- End Test Code ---

As non-root:

Version                                               Average
3.2.4                0.22992   0.2187    0.22333   |  0.22398
3.4.3                0.20305   0.20587   0.20523   |  0.20472
3.6.4                0.292087  0.306894  0.298228  |  0.29907
3.8.0                0.304033  0.305575  0.304689  |  0.30477
3.8.1                0.353086  0.347669  0.352126  |  0.35096
gui-release          0.435697  0.434251  0.440063  |  0.43667
  (no-gui-libs)      0.391251  0.386516  0.387915  |  0.38856
  (no-gui)           0.428424  0.42845   0.42787   |  0.42825
dev                  0.444153  0.441726  0.440012  |  0.44196
  (no-gui-libs)***
  (no-gui)           0.428423  0.428386  0.426818  |  0.42788

In summary:

Version               a=a+b;       a;       1;
3.2.4                0.88159  | 0.42259  |  0.22398
3.4.3                0.73678  | 0.29098  |  0.20472
3.6.4                0.90258  | 0.32309  |  0.29907
3.8.0                1.1275   | 0.39156  |  0.30477
3.8.1                1.0917   | 0.42879  |  0.35096
gui-release          2.7672   | 1.0572   |  0.43667
  (no-gui-libs)      2.3498   | 0.92489  |  0.38856
  (no-gui)           2.6708   | 1.0822   |  0.42825
dev                  2.7504   | 1.1063   |  0.44196
  (no-gui-libs)***
  (no-gui)           2.6890   | 1.1279   |  0.42788

and I've attached a plot of the above data to illustrate the trends.

Some more comments:

* As noted previously, 3.4.3 had the most efficient for-loop (not much 
better than 3.2.4 actually, if one looks at the red curve).  After that 
there is a slow trend upward in the consumption due to the for-loop.

* 3.4.3 also seems to have the most efficient memory/variable 
assignment.  There is an improvement from 3.2.4, but then decreased 
performance at 3.6.4 already.

* The decreased performance in looping (red line), that coincides with 
gui-release is--as Rik stated previously--about 10% to 15%, not the 
biggest portion of the lost performance.

* The increased consumption that coincides with the gui-release appears 
to be associated with the assignment or access of variables.  The 
missing big jump in consumption for the non-variable loop (the red 
curve) suggests the GUI itself is not the source of the problem.

Dan

for_loop_trends.png
Description: PNG image