Re: Simulating Individual Behavior

[Rick Riolo]

Jan,
I think evolutionary algorithms (EAs) can contribute
to the exploration of parameter value spaces, but I think
one should use such techniques with care and with eyes open.
Some of the issues include:
1. How do you make sure your definition(s) of fitness,
   or more generally, your implementation of selection
   pressures, will be the same as those that exist
   in the world you are trying to model?
   If they are not, then your model may evolve some
   parameter values but they won't reflect the underlying
   world "parameter" values.  
2. EA's are notoriously touchy.  You have to make
   choices about mechanisms and parameter values for the EA itself, 
   you have to make representation choices, and so on.
   They definitely don't come with any kind of guarantee that
   they will find optimal (or even near optimal)
   "solutions" (ie sets of model parameter values in this case).
3. Part of the reason for 2 is that the
   spaces one searches with EA's are almost
   always "complex", often with many many 
   "local hills" of pretty good solutions.
   Thus you might run the same algorithm from random
    starts 20 times and come up with 20 different
   "solutions" (parameter value settings).
   Then you have to choose amoung those,
    and of course there are all the other nearly equally
    good parameter settings you didn't ever see.
   How does one do that?  Usually you go back to
   whatever empirical data, first principles, etc,
   you know about domain in question...that is,
   you are (almost) back to where you started from!

Now I don't want to leave you with the impression
that I have it out for EA's in general or EA's in 
this context....I use them all the time!
They can help in the search for parameter values that
at least lead to desired behavior...but you still have
to make the case that the values found are 
plausible parameters values, given whatever other evidence 
you can bring to bear. 
I just think one has to be careful not ask or expect EAs 
to do more than they can do.

One interesting use of EA's in connection to model
parameter searches is exemplified by John Miller's
ANT paper (available as an SFI working paper):
96-03-011 Active Nonlinear Tests (ANTs) of Complex Simulation Models.
The basic idea is to use EA's to find weak or
questionable aspects of a model's design.
The important conceptual change is that rather than finding
some "one optimal parameter settings" one is looking for
*any* of the (perhaps many) parameter settings that
can lead to really bad model behavior.

Steven Bankes at RAND has also suggested similar
uses of EA's for various parts of "exploratory modeling."

 - r
 
Rick Riolo                       address@hidden
Program for Study of Complex Systems (PSCS)
1061 Randall Lab     University of Michigan
Ann Arbor MI 48109-1120
http://pscs.physics.lsa.umich.edu/PEOPLE/rlr-home.html

On Fri, 18 Apr 1997, Jan Kreft wrote:

> Date: Fri, 18 Apr 1997 15:05:25 +0100 (BST)
> From: Jan Kreft <address@hidden>
> To: address@hidden
> Subject: Re: Simulating Individual Behavior
> 
> Doug,
> 
> thanks for your comments. 
> 
> Doug Donalson wrote:
> 
> > Be careful of this method.  There are at least two
> > main problems you can run into.  The first is that you can
> > force fit a set of parameters to a model and get a result out 
> > that matches some experimental data but you have no guarentee
> > that you have any more that a pretty picture.  
> 
> The way to avoid such a force fit that I'm thinking about is
> optimizing/evolving the parameter set. Suppose that the parameter set is
> optimized with respect to the individuals' fitness (one or a few
> definitions of fitness or performance could be used). Then you compare the
> result of only those sims with optimized params with real data (in my case
> lab data, therefore the variation will be better known hopefully). (Real
> data also are the result of (evolutionary)  optimized behavior).  If you
> don't get a reasonable match the assumptions about the basic mechanism
> will be incorrect or lacking important points.
> 
> Does this make sense to you? Do you see a better way to get around the
> lack of appropriate, not theoretically biased quality data?
> 
> Simply setting parameters to experimental values beforehand could get you
> into serious problems also if a tiny deviation of a param has a large
> effect and it happens that that experimental value is just a bit wrong...
> 
> Jan.
> 
> > [...]
> > If you run a field experiment 20 times you
> > will get 20 results and hope that there is enough commonality
> > to get statistical power.  Therefore, the result you are
> > to which you are callibrating is just a sample from a statistical
> > distribution.  If that distribution has a wide varience then you
> > may have fine-tuned your model to an point far from the expected
> > behavior.  
> > 
> > A second trap is that if your assumption of the basic mechanisim
> > is incorrect you may fine-tune to a different system that has
> > the same result for that set of parameters.
> > 
> > Unfortunatly, I don't has a good set of procedures to eleminate 
> > these hazzards.  I am also not saying that this is a bad method,
> > it happens to be one I saw abused in an attempt to justify
> > the use of a differential equation based mode.
> > 
> > Doug Donalson
> 
> 
> 
>                   ==================================
>    Swarm-Modelling is for discussion of Simulation and Modelling techniques
>    esp. using Swarm.  For list administration needs (esp. [un]subscribing),
>    please send a message to <address@hidden> with "help" in the
>    body of the message.
>                   ==================================
> 


                  ==================================
   Swarm-Modelling is for discussion of Simulation and Modelling techniques
   esp. using Swarm.  For list administration needs (esp. [un]subscribing),
   please send a message to <address@hidden> with "help" in the
   body of the message.
                  ==================================