[Chicken-users] ok, I'm stumped

[Peter Keller]

Hello,

Maybe those of you here can help me understand how to do something. I am
quite thoroughly stumped.

So, I'm building an FFI to zlib.

The main(and only) structure you deal with in the zlib interface looks
like this:

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;typedef struct z_stream_s {
;    Bytef    *next_in;  /* next input byte */
;    uInt     avail_in;  /* number of bytes available at next_in */
;    uLong    total_in;  /* total nb of input bytes read so far */
;
;    Bytef    *next_out; /* next output byte should be put there */
;    uInt     avail_out; /* remaining free space at next_out */
;    uLong    total_out; /* total nb of bytes output so far */
;
;    char     *msg;      /* last error message, NULL if no error */
;    struct internal_state FAR *state; /* not visible by applications */
;
;    alloc_func zalloc;  /* used to allocate the internal state */
;    free_func  zfree;   /* used to free the internal state */
;    voidpf     opaque;  /* private data object passed to zalloc and zfree */
;
;    int     data_type;  /* best guess about the data type: ascii or binary */
;    uLong   adler;      /* adler32 value of the uncompressed data */
;    uLong   reserved;   /* reserved for future use */
;} z_stream ;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

Some functions that use it, look like this:
int deflateInit (z_streamp strm, int level);
int deflate (z_streamp strm, int flush);
int deflateEnd (z_streamp strm);
int inflateInit (z_streamp strm);
int inflate (z_streamp strm, int flush);
int inflateEnd (z_streamp strm); 
int deflateInit2 (z_streamp strm,
int deflateSetDictionary (z_streamp strm, const Bytef *dictionary, 
    uInt dictLength);
int deflateCopy (z_streamp dest, z_streamp source);
int deflateReset (z_streamp strm);
int deflateParams (z_streamp strm, int level, int strategy);
int inflateInit2 (z_streamp strm, int windowBits);
int inflateSetDictionary (z_streamp strm, const Bytef *dictionary, 
    uInt dictLength);
int inflateSync (z_streamp strm);
int inflateReset (z_streamp strm); 

Bytef is a typedef to an unsigned char* and z_streamp is a typedef z_stream
to a z_stream*.

Now, the way you're supposed to use this in C is to allocate in the heap
or on the stack the above z_stream object, fill in some fields, and then
pass it to the zlib functions.

For example, if I was decompressing, I'd do this:

void test_inflate(Byte *compr, uLong comprLen, Byte *uncompr, uLong uncomprLen)
{
    int err;
    z_stream d_stream; /* decompression stream */

    strcpy((char*)uncompr, "garbage");

    d_stream.zalloc = (alloc_func)0;
    d_stream.zfree = (free_func)0;
    d_stream.opaque = (voidpf)0;

    d_stream.next_in  = compr;
    d_stream.avail_in = 0;
    d_stream.next_out = uncompr;

    err = inflateInit(&d_stream);
    CHECK_ERR(err, "inflateInit");

    /* this is interesting, the zlib function adjusts things in the struct */
    while (d_stream.total_out < uncomprLen && d_stream.total_in < comprLen) 
    {
        d_stream.avail_in = d_stream.avail_out = 1; /* force small buffers */
        err = inflate(&d_stream, Z_NO_FLUSH);
        if (err == Z_STREAM_END) break;
        CHECK_ERR(err, "inflate");
    }

    err = inflateEnd(&d_stream);
    CHECK_ERR(err, "inflateEnd");

    if (strcmp((char*)uncompr, hello)) {
        fprintf(stderr, "bad inflate\n");
    exit(1);
    } else {
        printf("inflate(): %s\n", (char *)uncompr);
    }
}

Now, I don't understand how to construct an API for zlib because I don't know
how to make a z_stream structure that can be updated both in the scheme side
and the C side, and have garbage collection not foul anything up.

Here is a bad scheme translation of the above code, and what I'm striving
for it to generally look like:

(define test-inflate
;; compr and uncompr are u8vectors
;; uncompr and uncomprlen are integers
(lambda (compr comprlen uncompr uncomprlen)
    (let (  (err 0)
            (d_stream (make-z_stream))) ;; object using message passing

        (fill uncompr "garbage")
        ;; if the message has an extra argument, set! it, otherwise it is a ref
        (d_stream 'zalloc 0) 
        (d_stream 'zfree 0)
        (d_stream 'opaque 0)

        (d_stream 'next_in compr)
        (d_stream 'avail_in 0)
        (d_stream 'next_out uncompr)

        (set! err (inflateInit d_stream))
        (let loop ((done (and   (< (d_stream 'total_out) uncomperlen)
                                (< (d_stream 'total_in) comprlen))))
            (if (equal? done #t)
                #t
                (begin
                    ;; force small buffers
                    (d_stream 'avail_in 1)
                    (d_stream 'avail_out 1)
                    (set! err (inflate d_stream, Z_NO_FLUSH))
                    (if (equal? err Z_STREAM_END)
                        #t
                        (loop (and
                                (< (d_stream 'total_out) uncomperlen)
                                (< (d_stream 'total_in) comprlen)))))))

        (set! err (inflateEnd d_stream))
        (if (equal? (u8vector->string uncompr) hello)
            (begin
                (print "bad inflate")
                (exit 1))
            (print "inflate(): " (u8vector->string uncompr))))))

I'm quite happy to translate the message passing style object into
something else that I can somehow pass into the C API functions, but not
unnecessarily copying any memory and understanding how it interacts with
the GC is too much for my little mind to bear since C structures in the
context of FFIs in chicken is a black art.

You can find the zlib manual which is concise and to the point here(it 
is literally just the API with some description about each function):
http://www.gzip.org/zlib/manual.html

Any help would be greatly appreciated.

Thank you.

-pete