[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [gnugo-devel] FW: Owl tuning help needed
From: |
Trevor Morris |
Subject: |
Re: [gnugo-devel] FW: Owl tuning help needed |
Date: |
Tue, 15 Oct 2002 21:42:31 -0400 |
At 10:26 PM 10/14/2002 +0200, Gunnar Farneback wrote:
>Arend wrote:
>> This huge delta convinces me that we need to automatically watch for
>> performance regressions in the same way as we are already watching for
>> move choice regressions. (Just counting difference in owl nodes for
>> owl.tst just doesn't suffice, e.g. Also your numbers show so much
>> variation that any single test suite is not enough.)
>
>But I don't think we should try to add it to the basic awk script or
>start updating node numbers in the .tst files.
>
>A better solution would be to have a more high-level regress script
>which could do more detailed node counting and generate fancy reports
>of both results and node counts.
>
>What I really would like is a system that automatically runs
>regressions (including node counts) for every CVS checkin (or a batch
>of checkins if they come close together) and presents results on the
>web. This would not need to be as detailed and informative as Trevor's
>pages, but with history so it's easy to follow results over time.
>
>Possibly the Xenofarm project at
>http://pike.ida.liu.se/applications/xenofarm/ could be useful.
These stats are already computed by regress.pl
It creates an xml and trace file for each problem in the regression suite.
The xml files contains a whole slew of useful information, including a section
that looks like:
<COUNTERS
life_node="0"
owl_node="9602"
reading_node="1883407"
trymove="2522773">
You can see recent generated files at:
http://www.public32.com/pcolon/builds/gnugo/2002-10-10/gnugo/regression/html/
The HTML views just reformat this XML into a much easier to read format.
Running the regressions with regress.pl automatically on some trigger,
loading the results into a data warehouse, and analyzing the results should
not be too difficult.
-Trevor