[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: "Pace is nothing without guile"
From: |
Neil Jerram |
Subject: |
Re: "Pace is nothing without guile" |
Date: |
Sun, 13 Jul 2008 23:31:22 +0100 |
2008/7/13 Greg Troxel <address@hidden>:
> My immediate reaction is that test suites aren't good benchmarks because
> we will often want to add to test suites, while changing the benchmark
> invalidates previous data so we will not want to change the benchmark.
Yes, but...
> Now, if you mean to use the test suite as a collection of
> micro-benchmarks, so that we just have a rule that individual tests
> aren't modified without serious cause, but new ones can be added, then
> that makes senes.
Yes, this is what I meant. And I also think there will be enough test
cases not changing, over the long term, that we won't have to worry in
practice about changing a few here and there.
I've taken a better look now at the existing benchmark-suite - which
doesn't contain very many benchmarks, but does provide a sensible
discussion of timings, and useful library functions. I don't think it
will be very hard to somehow incorporate the test-suite into the set
of benchmarks.
Regards,
Neil