Re: Benchmarks (who has the fastest free VM)

[Michael Koch]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Am Dienstag, 8. Juli 2003 00:39 schrieb Mark Wielaard:
> Hi all,
>
> Before going to the Libre Software Meeting and LinuxTag
> <http://www.libresoftwaremeeting.org/>
> <http://www.linuxtag.org/2003/en/index.html>
> I wanted to know how the different free VMs were doing.
> So I did some micro benchmarks against them. I didn't have time to
> do more complete benchmarks and I couldn't get Orp and JRVM working
> (which is a shame since those are claimed to be the fasted VMs
> there are). So maybe someone with more time (or access to working
> Orp/JRVM VMs) can do some bigger tests.
>
> Orp didn't compile since it has some hardcoded values for glibc 2.1
> and glibc 2.2, but I am using glibc 2.3. And it needs several fixes
> to get it to compile with gcc 3.x. JRVM did compile, but needs a
> specific version of a proprietary VM to create the boot image. The
> resulting JikesRVM binary and image keep seg faulting however on my
> machine.
>
> The machine that ran the test had a Athlon 1600+ (1.4Ghz) and
> 768MB. It was lightly used and I don't claim to have done
> scientific benchmarking.  All times were measured with
> /usr/bin/time -p and the best of three runs is given. (Note that
> some tests measure elapsed time themselves so the time -p values
> are only for reference.)  The benchmarks were mentioned earlier on
> shudo.net. If someone has more suggestions please let me know.
>
> VMs used:
> kissme 0.0.30 (Jun 24 2003)
> SableVM version 1.0.8
> kaffe 1.1.x-cvs (checkout today)
> ikvm (Latest ikvmbin snapshot plus Mono JIT compiler version 0.25)
> gij/gcj 3.3.1 20030626 (Debian prerelease)
> (For gcj tests were compiled from source using just -O2)
>
> Looking at the results (see below) there are clearly three groups:
> the plain interpreters (kissme, gij and sablevm), the jitters
> (kaffe and ikvm) and the ahead of time compiler (gcj). The jitters
> are clearly much faster then the interpreters, and the ahead of
> time compiler is clearly much faster then the jitters.
>
> Interesting is that ikvm has results comparable with kaffe, but
> often takes more time "preparing" the classes. This can be clearly
> seen with the Linpack benchmark where the ikvm score and kaffe
> score are almost identical, but when looking at the real time spend
> you see that ikvm consumes much more processor cycles.
>
> SableVM recently got a "portable" jit compiler, it would be
> interesting to see how it matches up to kaffe and ikvm.
>
> Since gcj compiled code seems to be very fast but the gij
> interpreter is not very fast it might be interesting to combine gcj
> with one of the jitters or make one of the jitters be able to use
> precompiled binaries produced by gcj (for example to precompile all
> core classes used by the VM).
>
> Let me know if you have other benchmark/speed test stories.

It would be nice if you could put this onto a webpage for reference.
I currently search some performance comparisons for a german company 
which thinks about switching to gcj.


Michael
- -- 
Homepage: http://www.worldforge.org/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.1 (GNU/Linux)

iD8DBQE/Ck/zWSOgCCdjSDsRAgByAJ46vDXJ8gv7WkelDXbfBEp3bqyEhQCfeb7v
3xsv3D/B9ee370TXED2I1Ls=
=0xom
-----END PGP SIGNATURE-----