[Chicken-users] benchmarks

[Elf]

this is the third set of responses received regarding the fannkuch benchmarks.
responses have gone from snarky to snarkier to snarkiest.  im in favour of
setting up a benchmarks page off the main wiki page with results in the future.
thoughts?

-elf


---------- Forwarded message ----------
Date: Sun, 23 Mar 2008 03:26:28 +0000
From: address@hidden
To: address@hidden
Subject:
    =?UTF-8?B?W3Nob290b3V0LUJ1Z3NdWzMxMDY3M10gZGlmZmVyaW5nIHBlcm11dGF0aW9uIG9yZG
    VyIGlzIG5vdCBhIGJ1ZyBidXQgY2xhc3NpZmllZCBhcyBzdWNo?=

Bugs item #310673, was opened at 2008-03-22 10:46
Status: Closed
Priority: 3
Submitted By: elf ling (elf-guest)
Assigned to: Nobody (None)
Summary: differing permutation order is not a bug but classified as such
Category: Build and Measurement
Group: None
Resolution: Rejected


Initial Comment:
the scheme chicken benchmark is currently classified as 'error' because the 
permutation order differs, not because the result is incorrect.  this is an 
error in program accuracy checking.

----------------------------------------------------------------------

> Comment By: Isaac Gouy (igouy-guest)
Date: 2008-03-22 20:26

Message:
Obviously the algorithm is not /equivalent/ enough.

----------------------------------------------------------------------

Comment By: elf ling (elf-guest)
Date: 2008-03-22 19:14

Message:
the permutation order differs because an equivalent algorithm was used that 
does not penalise chicken with needless garbage collections.  as the algorithm 
for generating the permutations is NOT specified as part of the problem 
description, why is there a problem with this?

----------------------------------------------------------------------

Comment By: Isaac Gouy (igouy-guest)
Date: 2008-03-22 16:52

Message:
The first 30 permutations are output specifically so we can check that the 
permutation order is the same.

You were asked to diff your program output against expected output - that would 
have been enough for your to learn that your program was not acceptable.

----------------------------------------------------------------------

You can respond by visiting:
http://alioth.debian.org/tracker/?func=detail&atid=411002&aid=310673&group_id=30402