Re: [Swarm-Support] memory problems

[Marcello]

Thanks a lot for the reply. Following Paul's suggestion, I wrote a
method to make the computations I needed on the agents, so I do not need
EZDistribution anymore. 

In case someone might be interested in debugging the memory leak, I have
done some experiments with EZDistribution (and EZBin as well), and the
problem does not seem to be related with dropping the object: I found
that there is a huge memory usage increase even when only *one*
EZDistribution is created, and each agent only updates it. 

Specifically, I create a single instance of EZDistribution in
SwarmModel, which can be accessed by each agent via a method call. 
Now each agent calls the collection upon which EZDistribution operates,
the agent cleans up the collection and add new agents, updates the
distribution and forget about it. Even in this case, I can get my
program using up to 300mg in a little while.

CODE
in Swarm model (- buildObjects )

  somePeople=[List create: [self getZone]];

  somePeopleTab=[EZBin createBegin: [self getZone]];
  [somePeopleTab setTitle: "stats"];
  [somePeopleTab setGraphics:0];
  [somePeopleTab setBinCount: 5];
  [somePeopleTab setLowerBound: 0];
  [somePeopleTab setUpperBound: 5];
  [somePeopleTab setCollection: somePeople ];
  [somePeopleTab setProbedSelector: M(getTypology) ];
  [somePeopleTab createEnd];

//// other code

- getSomePeople {

return somePeople;
} 


- getSomePeopleTab {

return somePeopleTab;
} 


(In Agent.m)

  id observed;
  id group;

    group=[model getSomePeople];
    [group removeAll];
 
   //this fill up "group"
  for (i=0; i<10;i++) {
    while ([group contains: (observed = [self look])] ) ; 
     [group addLast:observed];
 }
  [[model getSomePeopleTab] update]; //if I remove this, no memory
problem occurs. With this line, the program grows (very roughly) about
1mg for each thousand calls .


// other unrelated stuff





Thanks again
marcello




On Tue, 2003-06-03 at 20:35, Paul E Johnson wrote:
> I agree that the drop on the EZDistribution object should give all the 
> memory back, but I don't think most people have tried to use that class 
> in this way and so, if there is a memory leak, we have not been alerted 
> to it yet.  I found a memory leak, which I still have not solved, in a 
> method that used probes to retrieve double-valued variables, and if you 
> are doing that, I expect you could see the same issue.
> 
> I think that you should redesign your class so that each agent only 
> creates one EZDistribution as an instance variable and then use it over 
> and over.  There is no need to create and drop at every time.
> 
> The other (better!) possibility is you should find a better way to 
> collect the information you want.  For example, are you just trying to 
> get a tabulation of how many agents there are in each group?  If so, 
> consider using a method like (off the top of my head, excuse typos)
> 
> 
> suppose there is an instance variable
> 
> int tabulator[6]; //need 6 because your example shows values from 0 to 5
>                    //not 5 binCount as you have;
> 
> and somewhere you initialize that to 0's
> 
> Then call a method that does the tabulation. This will be much faster 
> than creating a big old EZDistribution.
> 
> - tabulateDistribution
> {
>     id index = [group begin];
>     id anObject;
> 
>     for (anObject=[index next]; [index getLoc == Member]; 
> anObject=[index next])
>         {
>        int agentValue = [anObject getTypology];
>        tabulator[agentValue]++;
>          }
> }
> 
> 
> Or, if you just want to calculate an average or standard deviation, you 
> can do that more easily with EZGraph, I can show you how if you need.
> 
> Sorry, gotta run to a meeting.
> 
> 
> Here's my old email about the problem:
> 
> One of the guys who ran the Artificial Stock Market for a long long time 
> said it grew from 6meg of RAM to  300meg. I said you are crazy, but he 
> was right.
> 
> After trying every thing I know (dmalloc, etc), I could not find a leak 
> in ASM-2.2.1, but I have a strong hint that it is in Swarm itself.
> 
> Here is the source code.where you can see all this for yourself:
> http://artstkmkt.sourceforge.net/updates/ASM-2.2.1.tar.gz
> 
> There are some functions that I've used to get parameters. Here they 
> are, from BFParams.m:
> 
> id
> makeProbe (id obj, const char *ivarName)
> {
>    id probe = [VarProbe createBegin: [obj getZone]];
>    [probe setProbedClass: [obj getClass]];
>    [probe setProbedVariable: ivarName];
>    return [probe createEnd];
> }
> 
> double
> getDouble (id obj, const char *ivarName)
> {
>    id probe = makeProbe (obj, ivarName);
>    double ret = [probe probeAsDouble: obj];
>    [probe drop];
>    return ret;
> }
> 
> int
> getInt (id obj, const char *ivarName)
> {
>    id probe = makeProbe (obj, ivarName);
>    int ret = [probe probeAsInt: obj];
>    [probe drop];
>    return ret;
> }
> 
> There are many usages of getInt() and in BFagent.m, I had 2 usages of 
> the getDouble().
> 
> 
> After experimentation, I found the memory leak is caused by the use of 
> getDouble(), but not getInt().
> 
> You can check for yourself. In BFagent.m, change
> 
>   divisor = getDouble(privateParams,"lambda")*forecastvar;
> To:
>    divisor = 0.3*forecastvar;
> 
> Also change:
>    variance = getDouble(privateParams, "initvar");
> To:
>    variance = 4.00;
> 
> After that, the memory usage of the ASM stays steady for thousands and 
> thousands of steps.
> 
> That makes me think that the getInt() (the integer probe) is OK, but the 
> probe for a double has some issue.
> 
> So, where?
> 
> 
> There is still a memory leak if you just do like so:
> 
> 
> double
> getDouble (id obj, const char *ivarName)
> {
>   id probe = makeProbe (obj, ivarName);
>   //  double ret = [probe probeAsDouble: obj];
>    double ret = .5;
>   [probe drop];
>    return ret;
> }
> 
> Hence, it is not probeAsDouble: that triggers the leak. I reason 
> therefore the problem has to be in the creation of the probe for a 
> double valued ivar. maybe in VarProbe.m, the allocated memory for the 
> IVAR "probedVariable" is not getting dropped. But I don't see why it 
> would not be.
> 
> Aside from gazing at the source code, where I find nothing wrong, I 
> don't know how to attack this.
> 
> If you advise me, I will try.
> 
> -- 
> Paul E. Johnson                       email: address@hidden
> Marcello wrote:
> > Hi,
> > I have a swarm piece in Object C. It does everything using about 1% of
> > my 512mg of memory. Now I added one EZDistribution to each Agent (which
> > is a SwarmObject). The idea is that each agent creates a list of other
> > agents, passes it through a EZDistribution to get some statistics and
> > drop everything right after (see code). The problem is that when this
> > piece of code is in the program, the program sucks up a large amount of
> > memory, something like 5% of the memory each 100 calls to agents
> > (according to "top"). 
> > After some checks I know is the EZDistribution that creates the memory
> > problem, but I do not know how to solve it. Shouldn't my code create the
> > EZobject and dispose of it each time? 
> > Thanks for any help youcan provide
> > marcello
> > 
> > id a;
> > // here I create a "group" as a List. this does not create memory
> > problem.
> > 
> > 
> >  a=[EZDistribution createBegin: [self getZone]];
> >   [a setTitle: "stats"];
> >   [a setGraphics:0];
> >   [a setBinCount: 5];
> >   [a setLowerBound: 0];
> >   [a setUpperBound: 5];
> >   [a setCollection: group ];
> >   [a setProbedSelector: M(getTypology) ];
> >   [a createEnd];
> >   [a update];
> >   [a drop ];  //shouldn't this line dispose of "a"?
> >   [group drop];
> > 
> > thanks alot
> > mc
> > 
> 
> 
> 
> -- 
> Paul E. Johnson                       email: address@hidden
> Dept. of Political Science            http://lark.cc.ku.edu/~pauljohn
> 1541 Lilac Lane, Rm 504
> University of Kansas                  Office: (785) 864-9086
> Lawrence, Kansas 66044-3177           FAX: (785) 864-5700
> 
> _______________________________________________
> Support mailing list
> address@hidden
> http://www.swarm.org/mailman/listinfo/support
-- 
Marcello Gallucci (Ph.D)
Department of Social Psychology
Free University
Van der Boechorststraat 1
1081 BT Amsterdam (NL)
Tel. +31(0)20 4448846
Fax  +31 (0)20 4448921