Re: [patch] mem_node shrink #2

[Stefan Monnier]

>> Is that true?  Why is it the case?
>> On 64bit systems we can have buffers larger than 256MB.

> If buffer text is xmalloc'ed, (re)allocated region is not mem_insert'ed unless
> GC_MALLOC_CHECK is defined.

> If you run (buffer-substring (point-min) (point-max)) when current buffer's 
> size
> is 300M (on 64-bit system), the storage for resulting string is allocated with
> MEM_TYPE_NON_LISP and isn't mem_insert'ed too.

> But yes, it still has problems - on 64-bit system, (make-vector 50000000 nil)
> wants ~380M of MEM_TYPE_VECTOR, which isn't fit. So my stuff introduces vector
> size limits - (32M - 3) items on 64-bit and (64M - 3) on 32-bit. It looks
> acceptable for me, but I'm not sure about others.

> Is there any other code which may wants to allocate >256M ?

I don't think that saving 4bytes on 64bit systems is worth the trouble.
Also to lift the limit on 32bit systems, we just need to store size/8
instead of size.

So basically we want `size' to have (BITS_PER_EMACS_INT - 4) bits and to
hold (size >> GCTYPEBITS).  If USE_LSB_TAG is not set, then we shouldn't
shift (so the top bits with be truncated instead).

The justification for (BITS_PER_EMACS_INT - 4) is "so that combined with
the 1 and the 3 from the other two bitfields we get a whole number of
words".  And the justification for why this is correct will have to
explain that this assumes GCTYPEBITS==3 so that pointers and integers
only have BITS_PER_EMACS_INT - 3 significant values and stripping an
extra bit should still be OK because the size of the largest object will
be similarly halved by the use of signed integers (both for the size of
arrays and for the size of buffer text).

The only "problem" left is when USE_LSB_TAG is not set: this will limit
the max size of an array to (BITS_PER_EMACS_INT - 4) whereas in such
a case the byte-size of an array is limited by (BITS_PER_EMACS_INT - 1)
(but there can only be 1 such array because the start address needs to
fit within (BITS_PER_EMACS_INT - 3) :-)).


        Stefan