Re: parameter files

[Rick Riolo]

Ginger, 
This is a fair representation, and you raise some excellent points
about what these kinds of tools need to provide to be
able to handle the variety of sim situations we encounter.

As I mentioned, the current drone is missing some things I wish
it had, including some you mention:
- an algorithmic way to search the parameter space.
  The current drone requires the user to specify the
  parameters and sets of values to try.
  (it need not be a full n^m sweep...one can just pick
   and choose, but the user must do the picking and choosing)
- a better way to handle complex input specifications.
  drone can work with sims that require input specifications 
  of model structure (eg, a list of agents and "parameter settings" 
  for each agent, for global vars, etc),
  but it takes a bit of kludge to sweep over parameter
  settings in the individual agents, or to sweep
  over model structure.
- better and more general dispersal of jobs to
  multiple machines.   it does some of this, and its very
  handy, but there are contraints.

drone does have one control file that specifies
everything for an experiment, so that with that
one file one can reproduce the entire experiment
(ie start with same seeds and parameter settings).

Of course in truth, the real parameter processing
must be specified by the sim itself. 
Thus complex parameter files like those gecko needs
are really just passed on from the experiment control file
through drone and on to the sim, 
and its up to the sim to parse them.

- r

Rick Riolo                           address@hidden
Program for Study of Complex Systems (PSCS)
4477 Randall Lab                
University of Michigan         Ann Arbor MI 48109-1120
Phone: 734 763 3323                  Fax: 734 763 9267
http://www.pscs.umich.edu/PEOPLE/rlr-home.html

On Wed, 26 Aug 1998, Ginger Booth wrote:

> Date: Wed, 26 Aug 1998 14:10:52 -0400
> From: Ginger Booth <address@hidden>
> To: address@hidden
> Subject: parameter files
> 
> Thanks to Jan for neatly summarizing the Gecko approach!
> 
> I think it might be edifying, though, to mention how Drone and Gecko came
> into being as two separate approaches to the "same" problem, because
> they're not, quite.
> 
> Drone preceded Gecko's parameter file scheme (though not Gecko) by a
> short while, both during the beta days of Swarm. Both the Drone crew and
> myself were pushing for the Swarm team solve this instead of us. Given
> the effort, I think all parties were willing to take a close-enough
> solution and make it work, if only they could skip doing it themselves.
> The Swarm crew had neither the time, nor probably the confidence that
> any of us were sure what the right solution would look like.
> 
> Drone, at least in its incarnation when I first looked at it to borrow/
> steal for Gecko, was set up to be a batch job runner for parameter
> sweeps. Very useful, if you have a few parameters to explore, with
> more or less reasonable ranges to sweep.
> 
> But that's not what I needed. I needed:
> 
>  a. A control file which completely specified the experiment to be
>     run. The same control file, output from each run, must be good
>     input to reproduce that same run. Legibility is good. But more
>     importantly, it must be ONE: ONE control file, completely
>     specifying the experiment, the statistics observations, the
>     algorithmic parameter search, if any, the whole shebang. ONE.
> 
>  b. Even the kinds of agents that exist in a Gecko run are specified
>      in the parameter file. It needs to make 'em, set parameters on
>     'em, create observables on 'em, etc. There are a couple global
>     objects, too. Those are also parameterized.
> 
>     So the "parameter file parser" (the protocol reader) is actually
>     building the entire simulation tableau, and setting parameters,
>     and setting what passes in Gecko for a "probe".
> 
>     I never did (probably never will) create any mechanism for
>     building Gecko runs from a user interface. The only way to drive
>          it is from the protocol file.
> 
>  c. Parameter sweeps are lovely in R^2 or R^3 space. Gecko, alas,
>     may have 20-50 parameters. Any attempt at a sweep or global
>     optimization approach to that merits a weak giggle. So the
>     protocol marks which parameters are to be algorithmically tuned,
>     and where to start (and optionally another parameter to vary as
>     an offset from), and from there, may God go with you. No sweeps.
>     The only God (tuning algorithm) Gecko has so far is an Evolution
>     Strategy engine, fwiw. (Ingo Rechenberg, if you're interested--
>     it's an approach that works well with real-valued R^n spaces if
>     n >> 1.)
> 
> What Drone and Gecko both need, and do, is to specify:
>  - the random seed, so the same experiment can be reproduced (with
>    more observables, for instance)
>  - a number of iterations, since they're stochastic processes and a
>    single run doesn't say much.
> 
> (I hope none of the Drone crew feels misrepresented here. Y'all did a
> great job, and were kind enough to do it in a more generic way! :)
> 
> Anyway, for what it's worth...
> 
> Best,
>  Ginger
> 
> 
> 
>                   ==================================
>    Swarm-Support is for discussion of the technical details of the day
>    to day usage of Swarm.  For list administration needs (esp.
>    [un]subscribing), please send a message to <address@hidden>
>    with "help" in the body of the message.
>                   ==================================
> 

                  ==================================
   Swarm-Support is for discussion of the technical details of the day
   to day usage of Swarm.  For list administration needs (esp.
   [un]subscribing), please send a message to <address@hidden>
   with "help" in the body of the message.
                  ==================================