Re: [Chicken-users] max f32vector size

[Felix Winkelmann]

Clifford M STEIN wrote:

> It seems that the largest f32vector I can create is about 4.19M
> elements.  Is there any way I can create something larger?

Argh. That's a lot... Size of any string- or vector
like object is limited to 2^24 elements (that's the number of bits
used in the header to represent the length). Since f32vectors are
internally just byte-vectors, their size is not counted in elements,
but in bytes (so about 16M bytes come down to 4.19M elements).

So what to do...

You could create a normal generic vector, which can have up to
16M elements, but that needs too much storage and is not really
satisfying.

I would recommend allocating the vector using the FFI. This will
also result in much better performance, since the garbage collector
will not have to shove around the data during each GC. Since large
data-structures like this are normally long lived, freeing
them manually is not a problem (otherwise you can of course just
set a finalizer).

#>!
static void *allocate_float_vector(__fixnum size, float init, __bool initp)
{
  int i;
  void *ptr = malloc(sizeof(float) * size);
  float *fptr = (float *)ptr;

  if(initp) {
    for(i = 0; i < size; ++i) *(fptr++) = init;
  }

  return ptr;
}

static float float_vector_ref(void *ptr, __fixnum i)
{
  return ((float *)ptr)[ i ];
}

static void float_vector_set(void *ptr, __fixnum i, float n)
{
  ((float *)ptr)[ i ] = n;
}
<#

(define-record float-vector ptr length)

(define make-float-vector
  (let ([make-float-vector make-float-vector])
    (lambda (size . init)
      (let* ([init (:optional init #f)]
             [ptr (allocate_float_vector size (or init 0) init)] )
        (assert ptr "out of memory")
        (make-float-vector ptr size) ) ) ) )

(define free-float-vector
  (let ([free (foreign-lambda void "free" c-pointer)])
    (lambda (v)
      (free (float-vector-ptr v)) ) ) )

(define (float-vector-ref v i)
  (assert (<= 0 i (sub1 (float-vector-length v))))
  (float_vector_ref (float-vector-ptr v) i) )

(define (float-vector-set! v i n)
  (assert (<= 0 i (sub1 (float-vector-length v))))
  (float_vector_set (float-vector-ptr v) i n) )

(define v (make-float-vector 10000000 99.99))
(print (float-vector-length v))
(print (float-vector-ref v 9999999))
(float-vector-set! v 9999999 123.456)
(print (float-vector-ref v 9999999))

(read-line)

(free-float-vector v)


cheers,
felix