Re: [Chicken-users] two minor tweaks to runtime.c

[Alaric Snell-Pym]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 09/29/2011 04:51 PM, John Cowan wrote:
> Alaric Snell-Pym scripsit:
>
>> If the supposed performance improvement can't be benchmarked, then
>> it's pointless, as nobody will actually benefit from it. Any case
>> where somebody can benefit from a performance improvement can be
>> turned into a benchmark that consists of running the code that is sped
>> up, and timing it.
>>
>> Benchmarks are like unit tests; they are snippets of code that perform
>> some operation but, rather than testing correct responses, their
>> emphasis is on testing resource usage.
>
> Your clarification down-thread that a benchmark can be of any size makes
> this comparison rather otiose.  Nobody is going to have a benchmark
> suite that includes tests like these:
>
> With patch #1234, application 'foo' runs in an acceptable 18 hours
> rather than an intolerable 25 hours.  (Obviously the improvement has to
> be nonlinear.)

Actually, where I work we do! The full test suite takes all weekend - on
a cluster of fairly beefy hardware, running different bits in parallel.
But there's a hierarchy of tests and benchmarks. The correctness tests
we run on our laptops before committing code "to the trunk" take fifteen
minutes, and we run benchmarks in the five-minute range for quickly
checking the results of changes. The full suite runs only once a week...

ABS

- --
Alaric Snell-Pym
http://www.snell-pym.org.uk/alaric/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk6EmFQACgkQRgz/WHNxCGp2IwCfcofES+0R2BLmTBZ18wStt5Yf
/zUAn3Z0NYBhWzWrDodOF6+gPi3441k5
=DVDu
-----END PGP SIGNATURE-----