classpath
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Progress on a Classpath mauve suite?


From: Chris Pickett
Subject: Re: Progress on a Classpath mauve suite?
Date: Tue, 08 Mar 2005 17:23:39 -0500
User-agent: Debian Thunderbird 1.0 (X11/20050116)

Roman Kennke wrote:
Am Dienstag, den 08.03.2005, 10:34 -0600 schrieb Archie Cobbs:

Roman Kennke wrote:

Ah, now I'm understanding. I think maybe you mean the mauve HTML pages
that I once put together? This basically tested different runtimes
against the Mauve suite and formatted to Mauve output nicely into HTML
for displaying in a browser, so it is easily visible which tests fails
(red) and which don't (green). Is that what you mean? If yes, then I can
tell you that the nessecary scripts are not online ATM due to lack of
resources on my side, but probably soon will be available again on
developer.c.o. At least I hope so...

Something like that :-)


<snip>

Ok, these are good suggestions for improving my mauve scripts.


If we're going to make these HTML pages the "official" place for
VM implementors to look for this information, that's OK. I'd just
like us to decide and then do it, keep them updated, etc. This can
perhaps be automated, etc.


They actually are (or were) automated. Mauve has been run in a daily
cron job and the rendering of the results has been dynamic (on-demand).
A comparison of the failures/successes between the VMs should not be
that difficult to add.

I think this kind of automated testing/regression testing for Classpath VM's is a great idea.

If it's possible, some performance numbers would be interesting so we find discrepancies between there VM's there as well.

Having a VM that passes all passing Mauve tests (and fails all failing ones, although it seems highly unlikely that tests that are meant to fail might "accidentally" pass) could allow you to declare, "100% compatible with GNU Classpath v. 0.14," which I think is a cool kind of certification.

We discussed on IRC the need for a free benchmark suite (in particular, a free equivalent to SPECjvm98), and this is also something that one might envision built into a cross-VM testing framework.

Cheers,
Chris




reply via email to

[Prev in Thread] Current Thread [Next in Thread]