Re: A whiff of reality...

[Alex Lancaster]

>>>>> "RB" == Robert  <address@hidden> writes:

[...]

>> And, as both Marcus and I pointed out, the arguments the reviewer
>> made can be applied at every level of modeling, right down to the
>> chip design.

RB> Yes, but I really don't think that's what the reviewer meant.
RB> With every new level of abstraction, we take that leap of faith
RB> that the underlying complexity was properly implemented.  Most
RB> people feel confident with their C compilers (I do)....maybe even
RB> their RNGs (I trust Sven's ;-) but what about the way someone
RB> scheduled swarm events?  

It's generally far easy to understand how a modeller schedules events
in a Swarm model than in a `roll-your-own' simulation.  Remember that
the *user* is ultimately responsible for specifying the exact
scheduling _not_ Swarm (which merely _enables_ scheduling in a
well-defined way).  It's not as if Swarm somehow `adds' things behind
the scenes that user didn't specify or makes those decisions for the
user.  Which, incidentally, is what a lot of folks who have heard
about Swarm, but have never really looked at Swarm or written a single
Swarm simulation often seem to believe.  In other words, they think of
Swarm more as a closed-system like SimCity, where you merely tweak
parameters, than a specification platform like Mathematica or Matlab.

So I think, the question then really becomes, "do I trust that the
underlying activity library implementation of scheduling", and I
think, based on the number of reimplementations of CA-like models in
Swarm (like SugarScape) against third-party implementations that the
implementation is pretty robust.

RB> If I was reviewing a model paper, I would not be too concerned
RB> with the possible compiler bugs, but I sure as heck would want to
RB> know every detail about the scheduling of events.  Now look at
RB> typical modeling papers...can you reproduce what they did? Not
RB> usually...there's hardly ever enough information. So there's lots
RB> of room for creative interpretations.  And the situation is far
RB> worse in areas where models are being used to make management
RB> decisions.  In these cases, the model user sometimes only knows
RB> that this model grows a tree...no idea how.  I believe the
RB> reviewer had this in mind and not chip design.

Understood, but (see below) I still believe that a generic platform
helps the cause of reproducibility/readability of a model, more that
hinders it.  

Contrast:

"send `grow' to each `treelist' at timesteps of interval 2"

to:

"for every timestep
 if timestep modulus 2 is zero
  then 
    for every element in the `treelist'
     send `grow' to that element"

Both are entirely equivalent constructions, but one is clearly closer
to the domain of ABM, and one is clearly more susceptible to
misspecification by the modeller than the other.

>> (Remember the PII(?) math error that Intel tried to ignore?)
>> Science could not advance if everyone had to start from scratch.
>> We just have to be careful to make sure we use the correct tool for
>> the correct job.

RB> Yes, of course....but there is a difference between trusting a
RB> tested chip (ignoring Chipzilla's fiascos) and trusting someone's
RB> tree growing algorithm.

Actually, I don't think there really is a difference in principle.  If
the output algorithm is well-specified (i.e. formally or
mathematically) then I don't see any reason to trust it any less.
It's simply a difference of degree and not kind.

The point made by several posters here that you already trust the
compiler, the run-time, the operating system, the chip, is totally on
the mark, so why not an ABM platform (which after all is logical
extension of the same chain of technology)?  

It's funny, because I've had this kind of discussion with people face
to face sometimes, and often when I explain what Swarm's all about, I
get the kind of macho response (more often from physicists for some
reason): "well, *I'd* never trust an algorithm [meaning a Swarm
object] I didn't write".

I simply reply: "well you trust the compiler, don't you?" and normally
they kind of fall silent for a while.  For some reason these folks get
stuck thinking that writing something in C is somehow more virtuous
and "real" than using higher-level `chunked' functionality.  To
suggest that should agent-based modelling adopt a programming language
like C (and a not very expressive one at that) as the appropriate
level of abstraction, it's just absurd.

Of course, it means that it becomes even important that the `chunked'
functionality is extremely well-specified and well-implemented and
completely transparent (i.e. open-source), but that's more a matter of
good documentation and software engineering than any intrinsic issue
problem in modelling.  I think that folks who are criticising the
higher-level approach have been burnt by `pre-fab' packages that don't
specify this information and are generally closed-source proprietary
shrink-wrapped `solutions', so you can't *ever* see "what's under the
hood".  In these cases, I would 100% agree with the critic.  The point
here is that Swarm and Swarm-alike platforms are trying to address
precisely the problems that the pre-fab solutions hide.  One way to
think of what Swarm could ultimately become (it's clearly not there
yet) is as a sort of compiler for an agent-based modelling language.

I'd argue that bringing the language of event-driven simulation and
agent terminology to the fore (i.e. into the user space) is actually
making it *more* likely to write `correct' and `validatable'
simulations than if you write things from scratch.  In specifying
things like `ActionGroups', in Swarm, forces you to be *more* explicit
about things like update order than if you were to "roll your own".
It certainly makes it easier for others to understand the structure of
your model.

All think this discussion ultimately means, is that Swarm and
Swarm-like tools are still a little ahead of their time.

I've had this discussion enough times, that I'd really be interested
in collaborating with some people and writing a pure methodology paper
addressing these issues.  So once and for all, I could say to the
critics: "well, read our paper".  Anyone interested?

Alex
-- 
Alex Lancaster * address@hidden * www.santafe.edu/~alex * 505 984-8800 x242
Santa Fe Institute (www.santafe.edu) & Swarm Development Group (www.swarm.org)


                  ==================================
   Swarm-Modelling is for discussion of Simulation and Modelling techniques
   esp. using Swarm.  For list administration needs (esp. [un]subscribing),
   please send a message to <address@hidden> with "help" in the
   body of the message.
                  ==================================