Re: [Chicken-users] Memory allocation and limits in chicken scheme programs

[Peter Bex]

On Wed, Dec 13, 2017 at 05:16:52PM -0700, Matt Welland wrote:
> Much better with 4.13. Thanks. I have it working up to 128G. I don't
> understand the heap and will do some more reading but am I correct in
> assuming that the size of the heap will determine the largest data
> structure I can have in memory?

The largest data structure is limited by both the maximum heap size
and the representation of objects: we use WORDSIZE - 8 bits to represent
the size of an object.  For bytevectors, strings and the like (this
includes bignums on CHICKEN 5, unfortunately) we store the size in bytes,
for Scheme block objects like vectors, records and such, we store the
size in words.

On 32 bit machines this is a serious restriction, on 64 bit machines
the heap will usually be the limiting factor.

If you're really interested in the gory details, see my blog post:
http://www.more-magic.net/posts/internals-data-representation.html

> I base this on seeing the VIRT column in
> htop plateau at 128G when I used -:nm128G.

Like Kon said, we use a Cheney style garbage collector, which involves
two semispaces between which objects get copied on GC.  So the total heap
size you specify is shared between these two halves.  This means the
effective size available to your program is half the size you specify
on the command line via the -:hm option.

Cheers,
Peter

signature.asc
Description: PGP signature