Re: Compare Swarm with Repast

[Gary Polhill]

>Dear Gary:
>
>Please consider making a measurement of Heatbugs with my optimized 
>HeatSpace class:
>
>http://lark.cc.ku.edu/~pauljohn/Swarm/MySwarmCode/HeatSpace.m 
>
>and report back. All you need to do is drop that file in over the old one.

The difference between your HeatSpace and the one that ships with the SwarmApps 
doesn't come out until you set the refresh rate high enough. (That is, all 
measurements using Obj-C Swarm with your HeatSpace are pretty much the same as 
those posted in my earlier message.)

It then occurred to me that maybe setting the refresh rate to 100 was a better 
way of measuring background performance than minimising the window. Here are 
the cycles per minute:

Sun:

JavaSwarm: 4900   (!)
ObjCSwarm (Standard HeatBugs): 4500
ObjCSwarm (Paul's HeatBugs): 6300

PC:

JavaSwarm: 2700
ObjCSwarm (Standard HeatBugs): 6900
ObjCSwarm (Paul's HeatBugs): 9400

Looks like with your modifications you now get about a 50% improvement in speed.

I tried hacking the HeatBugsModel.java that comes with RePast to make it do a 
refresh rate. My efforts are attached, but since I really don't know what I'm 
doing maybe it's not fair to compare the results I got (1700 cycles/min on my 
PC, 1400 on the Sun FWIW) with those from Swarm above.

I've no idea how the JavaSwarm managed to beat the standard ObjCSwarm on the 
Sun but not on the PC... There seem to be an awful lot of other factors coming 
in to play in determining the time than just programming language and model 
algorithm here, which makes interpreting these benchmark results a real 
challenge! It's kind of important for us too -- we often run our model many 
thousands of times, comparing parameter settings. If the model runs three times 
slower on one platform and modelling environment/language than another, that's 
3 months to get results rather than just 1. From the results I've got today 
it's still not really clear what the cost is (in batchmode) of switching from 
Obj-C to JavaSwarm (anything from 1.5x slower to 3x slower), and, were we to 
consider it, from JavaSwarm to RePast (anything from 1.5x faster to 3x slower).

There are version differences across the platforms I've been benchmarking on, 
so for the sake of thoroughness:

Sun: Solaris 8, Dual 750MHz sparc, 2.5G RAM. Swarm testing-2001-12-18, 
Repast-1.4 both using Java 1.3.1_01. heatbugs-2.1.1 (ObjCSwarm), 
jheatbugs-2001-03-28 (JavaSwarm), and in Repast I used the heatbugs that came 
with repast-1.4.tar.gz

PC: Windows 2000, 1.6GHz intel, 512M RAM. Swarm-2.1.1 using Kaffe 1.0.5, 
Repast-1.4.1 using Java 1.3.1_02. For JavaSwarm I used jheatbugs-2.1, for 
ObjCSwarm, I used the heatbugs off the Swarm 2.1.1 CD (installing on my hard 
drive first), and for Repast I used the heatbugs that came with 
repast-1.4.1.zip.

Gary



Gary Polhill
Research Scientist
The Macaulay Institute
Craigiebuckler
Aberdeen AB15 8QH
UK
Tel: +(44) (0)1224 498200 Ext 2238
Fax: +(44) (0)1224 311556
e-mail: address@hidden
http://www.macaulay.ac.uk/
http://www.macaulay.ac.uk/fearlus/

HeatBugsModel.java
Description: Text document