Re: [Chicken-users] spock tests

[Marc Feeley]

On 2012-09-08, at 8:28 AM, Felix <address@hidden> wrote:

> Hey, folks!
> 
> Many, many thanks for testing so pervasively. This is indeed good
> news, as it shows that the stuff generally works. As you all reported,
> some test-cases fail, but this is known and caused by minor bugs or
> R5RS incompatibilities that result from the way Scheme is mapped onto
> JS semantics in this translator. Now that we know that Spock is
> basically working, I can continue improving it. The performance is not
> that great but sufficient (and in fact doesn't look too bad when
> compared with Scheme2JS or Gambit's forthcoming JS backend, in
> particular when continuations are used heavily). Also, Spock is more
> geared towards dynamic code generation than for doing ahead-of-time
> compilation of heavyweight programs.
> 
> Thanks again for your assistance!
> 
> 
> cheers,
> felix

In my experiments (updated since we last talked to the most current version of 
Chrome, Firefox and Safari) I have found that the performance of Spock (and 
Scheme2JS) varies a lot between JavaScript VMs.  For example Nitro (Safari 6.0) 
runs JS code generated by Spock about 20 to 30 times slower than V8 (Chrome 
21.0.1180.89).  I have attached below the table of results from my Scheme 
workshop paper.  The programs ctak, contfib30, btsearch2000 and threads10 use 
continuations heavily.

The paper will be available on the Scheme workshop web page.

Marc


 Execution times using V8 (Chrome 21.0.1180.89):

                Gambit-JS        Scheme2JS             Spock
 fib35               .80        1.54    1.9x        2.40   3.0x
 nqueens12           .72         .76    1.1x        2.33   3.3x
 oddeven             .83        1.92    2.3x        5.62   6.8x
 ctak                .18       17.64   95.9x         .59   3.2x
 contfib30          1.17      106.01   90.9x        3.38   2.9x
 btsearch2000       1.35       25.40   18.8x        7.93   5.9x
 threads10          1.34       24.68   18.5x        4.40   3.3x


 Execution times using JagerMonkey (Firefox 15.0.1):

                Gambit-JS        Scheme2JS             Spock
 fib35              1.07        7.66    7.2x       20.73  19.4x
 nqueens12          1.49        2.62    1.8x       12.97   8.7x
 oddeven            1.09        1.93    1.8x       33.91  31.1x
 ctak               1.25       30.38   24.3x        1.86   1.5x
 contfib30          5.88      156.79   26.7x       10.61   1.8x
 btsearch2000      11.19       27.16    2.4x       16.54   1.5x
 threads10          6.97       34.36    4.9x       22.11   3.2x


 Execution times using Nitro (Safari 6.0):

                Gambit-JS        Scheme2JS             Spock
 fib35              1.32        6.53    4.9x       84.69  64.2x
 nqueens12          1.31        1.89    1.4x       45.15  34.4x
 oddeven            2.52        1.55     .6x      143.97  57.1x
 ctak                .30       52.88  177.4x        4.18  14.0x
 contfib30          1.60     2311.69 1443.0x       25.98  16.2x
 btsearch2000       3.46      102.98   29.7x       92.02  26.6x
 threads10          3.03       74.21   24.5x       74.88  24.7x