Re: mauve results posted nightly

[Brian Jones]

Stephen Crawley <address@hidden> writes:

> Maybe a Sun JDK 1.4.x for reference?

It's possible, if it helps.  I may set this up under tinderbox.

> If you have time to work on this, it would be a good idea if the web page 
> and results defined the testing methodology, to aid people in reproducing
> the test results.

Yes, I have the scripts if anyone wants them... but they aren't
refined enough to checkin to cvs I think.  

> * The log report should include datestamps for the latest sync with
> Classpath, Kissme and Mauve CVS repositries.  

Basically the date on the comparison files represents the approximate
time of the CVS sync.

> * the web page should document the hardware and OS platform/version, 
> the C & Java compilers and versions, and versions of key libraries used 
> (e.g. glibc).

My home box, Red Hat 8.0 GNU/Linux, jikes 1.17, glibc 2.2, linux
2.4.18.  I don't have anything else but Windows.  I think if I get
tinderbox setup then people with other configurations could run a
client there to report those results.

> * The Mauve "keys" file used should be linked to the page, or available
> some other way.  It should be version controlled, as it will surely change
> over time.

It's in Classpath CVS, mauve-classpath.

> Here are some other suggestions:
> 
> *  The web page could explain that there are many possible causes of test
> "failures"; e.g. Classpath bugs or incomplete feautures, Kissme bugs,
> Mauve test case bugs, and (even) Sun JDK bugs and incompatibilities.
> [Also, note the possibility of network failures, and failures caused by
> non-atomic CVS commits.]

It could, but this (nightly Mauve) is an end product for developers,
not users.  You already know these things and I don't have to explain
it to you.

> *  The nightly test results should be archived, with the archive ideally
> accessible via the web page.  It would be very nice if the web page
> could graph the numbers of tests passing and failing over time!! But
> that could be a lot of work to set up.

Yeah, I think of lot of what you want is tinderbox.

Brian
-- 
Brian Jones <address@hidden>