[Chicken-users] Passing zero-length (sub-)buffers to C functions

[Florian Zumbiehl]

Hi,

I am looking for a clean way to pass arbitrarily sized buffers to C
functions, where "arbitrary" in particular includes zero-length
(sub-)buffers.

As far as i can see, the mechanisms suggested to be used for passing
buffers to C functions special-case zero-length buffers to execution
failures, and thus break generality for code using the abstraction in a
natural way.

As I understand it, the recommended way to pass a gc-able buffer to a C
function is by wrapping it in a locative--in particular, if one wants to
pass a sub-range of the underlying data struncture (a "sub-buffer"), by
specifying an offset to make-locative. In addition, one may also need to
convey length information about the buffer to the C function, of course,
but that is supposed to happen via some unrelated mechanism and thus is
outside the scope of the following consideration.

The problem with that approach is that (non-weak) locatives apparently are
(also) meant to be references/pointers to objects, which is to say, are
meant to always be dereferencable to the one specific object they are
pointing at, or so #chicken tried to make me believe.

A buffer, though, does not necessarily have to contain any objects, as it
is perfectly reasonable to have zero-length buffers--which, thus, obviously
can not be represented by something that points at an object (in the
buffer).

So, the requirements of representing "a pointer to an object" and "a
pointer to a buffer" are at odds with one another, but still, it seems like
one is supposed to use the same data type for both in Chicken!?

Here is a trivial example that would seem like a natural way to wrap the
write() syscall (ignoring error handling for the moment) that fails due to
this mismatch:

| (define (sys-write fd buf)
|   ((foreign-lambda int "write" int c-pointer int)
|     fd
|     (make-locative buf)
|     (string-length buf)))
| 
| (sys-write 0 "hello world\n")
| (sys-write 0 "")

Which gives this output:

| hello world
| 
| Error: (make-locative) out of range

Another common situation where this happens is if you fill up a buffer
incrementally over multiple calls to some C API, each time passing the
pointer to the remaining free space. That's also how I stumbled upon this:

OpenSSL's EVP_CipherFinal_ex() expects a pointer where it puts any
remaining output that's produced by padding and encrypting the last partial
plaintext block that OpenSSL holds in its internal buffer. The natural way
to use that is by adding the amount of output data received so far to the
pointer to the start of the destination buffer, which gives you a pointer
to the remaining free space. Now, if you are using a stream cipher or a
block cipher with some stream cipher mode, there is zero remaining free
space, but there is also zero remaining output, so that is still perfectly
compatible with this approach: You pass the pointer that points "one past
the end" of the destination buffer (which is guaranteed to be legal in C,
you just may not dereference it), and EVP_CipherFinal_ex() does not
dereference it, and everything is fine.

Just my naive Chicken wrapper failed because it didn't special-case this
zero-length buffer and make-locative complained when it tried to obtain a
locative that points to the zero-length remaining free space.

So ... well, I don't really know where to take it from there. Maybe I am
just completely mistaken and locatives are the wrong tool for the purpose?
In that case, pointers to the correct approach would be very much
appreciated. Otherwise, I might have some more suggestions about useful
semantics for such a mechanism, but I'll leave it at that for now.

Regards, Florian