Re: experiment management

[pauljohn]

We had a nice go-around on this in the mailing list about 2
months ago. Fred Wan was looking at ways to do it, and he posted
several questions and descriptions of his experience.  I believe
in the end he came around to my way of thinking.

My recommendation:

learn how to pass command line arguments to your program and run
in batch mode.  Use an external program to manage the
repetitions.  I have used Drone and think it is good.  Because
my windows users could not get Drone to work, I created a perl
script that can do the replication and parameter sweeps (for an
arbitrary number of variables).  That is here:
http://lark.cc.ukans.edu/~pauljohn/Swarm/MySwarmCode/replicator.pl

I wrote up a little tutorial for Objective-C users who want to
pass command line 
arguments and may also want to watch the simultion in GUI mode. 
That is here:

http://lark.cc.ukans.edu/~pauljohn/ps909/Agent-Based_Models/simpleObserverBug3-6.tar.gz

I don't get any feedback one way or the other about this.  I
take that as positive, because when things are fouled up, I get
plenty of negative feedback :)

On the Java side of the equation, I believe replicator.pl can be
adjusted and Sven Thommesen posted an excellent memo on argument
passing from the command line and you can find a link to it in
the SwarmFaq.

The alternative approaches that some have considered.

1.  In the swarm tutorial, there is a simulation that repeats
itself with benefit of an experiment manager class.  Look at
simpleExperBug.  That one has the Swarm Gui controlling a
wrapper class that spawns simulations. You don't get Gui display
of the action going on in a simulation.  I don't see much point
in this approach, as a result, and think the batch mode is
better.  And if I were at a meeting about the tutorial, I'd vote
in favor of a motion to remove this component.

2.  I worked pretty hard to enable Gui display in a context like
that.    The last working model is this one:

http://lark.cc.ukans.edu/~pauljohn/Swarm/MySwarmCode/repeatingHeatbugsParameter-2.0.1x.tar.gz

This creates an experiment manager, and the "next" action spawns
a simulation that  you can watch and interact with.  There is
some monkey business (problems I am not taking time to fix) with
Swarm itself that Marcus Daniels explained. The problem causes
some crashes when you start/stop/start the individual runs of
the model.  I am not digging into this more because I think the
approach I recommend above is the best.

Until our recent go-around on this, I had forgotten that Steve
Railsback had emailed me with a variant of the simpleExperBug
approach that worked in Objective-C.  I remember it was a model
of fish movement in a river that had an absolutely beautiful
picture with it.  I think if Steve is reading this message, he
will probably announce for us the URL of his code, as I seem to
have lost it.

I will update the FAQ with my opinions here.  To the best of my
knowledge, I've given a full account of the differing points of
view.

Charlotte Bruun wrote:
> 
> Dear List,
> 
> Is there anything new on experiment management?. At the FAQ page there is
> an old discussion on how to repeat experiments (for example with different
> random seeds), but this is from 1998. How do users handle this problem in
> 2001?
> 
> Charlotte Bruun



-- 
Paul E. Johnson                       email: address@hidden
Dept. of Political Science           
http://lark.cc.ukans.edu/~pauljohn
University of Kansas                  Office: (785) 864-9086
Lawrence, Kansas 66045                FAX: (785) 864-5700

                  ==================================
   Swarm-Support is for discussion of the technical details of the day
   to day usage of Swarm.  For list administration needs (esp.
   [un]subscribing), please send a message to <address@hidden>
   with "help" in the body of the message.