Re: [Chicken-users] Segfault with large data-structures

[Jim Ursetto]

On Feb 2, 2013, at 3:46 PM, Kristian Lein-Mathisen wrote:

> I'm getting the same result here, when I run it through csc. When I run it 
> through csi, though, it never seems to finish - is the task that big? I had 
> to kill it after 2-3 hours.

It's a hash table with 250,000 entries and 1,000 items per entry.  That's going 
to be tens of gigabytes for heap (memory usage at 40,000 nodes just before it 
dies is about 2GB on my machine).  It's 250,000 node inserts (O(V)), 250 
million edge inserts (O(V*E)), and 125 billion operations (O(V*E^2)) to 
traverse the edge lists (since you are using member).  To me that is pretty big.

csi doesn't crash but appears to be stuck after 40,000 nodes inserted, the same 
place that csc crashes maybe.  It's not swap related, I had plenty of memory, 
so something is maybe up with the hash table or GC.  I can't explain why csi 
does not crash.

This machine is 64-bit OS X with 16GB main memory and no obvious limit to the 
process data segment.

Probably stupid shot in the dark: what happens when the Chicken heap size 
exceeds 4GB on a 64-bit machine?

Jim