[MIT-Scheme-devel] allocation bookkeeping for primitives

[Taylor R Campbell]

Some primitives, such as some of the bignum primitives, do multiple
separate GC checks for allocations, say for N words the first time and
M words the second time.  Suppose the N-word allocation succeeds, but
the M-word allocation fails, so that the primitive requests M words --
not N + M words -- from the GC; further suppose the GC successfully
recovers at least M but fewer than N + M words.

In this case, the runtime won't try to abort to the nearest REPL and
run the secondary GC daemons, because it thinks that enough space was
recovered; instead, the system will restart the primitive, which may
again successfully allocate N words and fail to allocate M words, and
repeat the process, wedging Scheme unrecoverably.

These primitives could be modified to approximate an upper bound on
the number of words they will allocate and do a single GC check before
allocating anything, but it's not always straightforward to do this.
(This is the approach Scheme48 takes.)

What if, instead, for every primitive invocation, we count the number
of words allocated?  That is, in PRIMITIVE_APPLY_INTERNAL, just before
calling the primitive, we store 0 in some counter C, and increment C
every time we advance Free.  Then REQUEST_GC (N) will request C + N
words, not just N, and we'll never loop as above.  Finding every place
we advance Free should be easier than modifying each primitive to
approximate an upper bound on the amount of space it could allocate.

This doesn't handle allocations in compiled code or in the compiled
interface.  I have always been worried about compiled code, which
doesn't bound the amount of space that can be allocated between heap
checks, but preventing allocation loops in primitives is a good start.
Also, it would be nice if the allocation failure conditions of
edwin/utils.scm could be moved safely into the runtime and used for
all allocation failures (with the existing abort->nearest behaviour if
there isn't enough space to signal an allocation failure condition).