wip: cdata module for dealing with C types in bytevectors

[Matt Wette]

Hi All,
I have been working on a module, an alternative to Scheme Bytestructures, say, that
can be used for cross-platform development.  It's still in progress but I am pretty far along.
I thought I'd share some aspects of the design.  I am currently working on ctype-equal? and
adding test code that generates, compiles and calls C code.

Matt

Introduction
************

Matt Wette
June 2024

Introduction
============

The ‘cdata’ module, with its partner ‘arch-info’, provides a way to work
with data and associated types originating from C libraries.  It supports
non-native machine architectures using a global ‘*arch*’ parameter.
Size and alignment is tracked for all types.  There are type classes for
all C type classes: base, struct, union, array, pointer, and function.
Beyond size and alignment, base type objects carry a symbolic tag to
determine the appropriate bytevector procedures: there is no type
aliasing.  Support for C base types is handled by the ‘cbase’ type
object generator which converts them to underlying types.  For example,
on a 64 bit little endian architecture, ‘cbase’ would convert
‘uintptr_t’ to a base class ctype with underlying type info symbol
‘u64le’.

   The module is hopefully easy to use.  One uses the procedures
‘cbase’, ‘cstruct’, ‘cunion’, ‘cpointer’, ‘carray’, and ‘cfunction’ to
generate c-type objects, and then ‘make-cdata’ to generate C data
objects.  Access to component data is provided by the ‘cdata-ref’
procedure and mutation is accomplished via the ‘cdata-set!’ procedure.
With respect to Guile’s ffi interface, one can use ‘ctype->ffi’ to
convert to FFI type specifiers required for use of
‘foreign-library-function’.

   Example:
     (define t1 (cstruct '((tv_sec long) (tv_usec long))))
     (define gettod
       (foreign-library-function
        #f "gettimeofday"
        #:return-type (ctype->ffi (cbase 'int))
        #:arg-types (map ctype->ffi (list (cpointer t1) (cpointer 'void)))))

     (define d1 (make-cdata t1))
     (gettod (cdata-ref (cdata& d1)) %null-pointer)
     (format #t "time: ~s ~s\n" (cdata-ref d1 'tv_sec) (cdata-ref d1 'tv_usec))
     time: 1719062561 676365

Handling Machine Architectures
==============================

Needs love ...

     (define tx (with-arch 'riscv64
                  (cstruct '((a long) (b int)))))

Constructing Types
==================

The procedures used to create C types are described in the following.
Type construction is machine architecture (i.e., parameter ‘*arch*’)
dependent.

 -- Procedure: cbase name
     Creates a C base type, given the symbolic NAME for that type (e.g.,
     ‘unsigned-int’).

 -- Procedure: cstruct field-list [#:packed? pp]
     Creates a C struct with field list in the form ‘((name type) ...)’
     where NAME is a symbolic name (or ‘#f’) and TYPE is a c-type.
     Anonymous structs (and unions) are specified using ‘#f’ for the
     field name.  For packed structures add ‘#:packed #t’.

 -- Procedure: cunion field-list
     Creates a C union with field list in the same form as ‘cstruct’,
     except bitfield types are not allowed.

 -- Procedure: cpointer type
     Create a pointer type referencing TYPE.  A non-negative selector
     can be used with this type to generate an increment.  Say you have,
     in C, the variables ‘x’ defined from ‘int *x;’ which represents an
     array in memory.  Then the _expression_ to reference the fourth
     element is ‘x[3]’ or, equivalently, ‘*(x+3)’.  In Guile, this
     _expression_ would be ‘(cdata* (cdata-ref x 3))’.

 -- Procedure: carray type length
     Create an array of TYPE with LENGTH.  If LENGTH is zero, the array
     length is unbounded (so be careful).

 -- Procedure: cfunction ret-type arg-types
     This is work to go.

 -- Record: cbitfield type width
     This type is only created internally in the ‘cstruct’ procedure,
     but a ‘cdata-sel’ operation can yeild a result of this type which,
     when provided as argument to ‘cdata-ref’ or ‘cdata-set!’, will do
     the right thing.
          (define st (cstruct '((a int) (b short 3))))
          (define sd (make-cdata st))
          (define sd.b (cdata-sel sd 'b)) ;; => #<cdata 0x12345678 bitfield>
          (cdata-set! sd.b 3)
          (cdata-ref sd.b) => 3

   As a special case to deal with void pointers, the _expression_
‘(cpointer 'void)’ can be used.  Data object of this type would be
converted to Guiles ‘pointer’ type by the procedure ‘cdata->ffi’.  The
_expression_ ‘(cbase 'void*)’ will generate the associated integer type
used for pointers.

Working with Data
=""

These procedures can be used to manipulate data.  They are not dependent
on the global ‘*arch*’ parameter.

 -- Procedure: make-cdata type [value]
     This procedure creates an object of type ‘<cdata>’.

 -- Procedure: cdata-ref data [tag ...]
     Return the Scheme value for cdata.  Since we provide no equivalent
     for structures in Scheme, cdata should have type class base,
     pointer, or array.  To get the value of a struct field, you should
     use the following form:
          (cdata-ref (cdata-ref struct-data 'a 'b 'c))

 -- Procedure: cdata-set! data value [tag ...]
          (cdata-set! struct-data 1 'a 'b 'c))

 -- Procedure: cdata-sel data tag ...
     Return a new ‘cdata’ object representing the associated selection.
     For example,
          dat1 -> <cdata 0x12345678 struct>
          (cdata-ref dat1 'a 'b 'c) -> <cdata 0x12345700> f64le>
     TODO: do we need something to return ‘(values bv ix ct)’

 -- Procecure: cdata* data
     to be documented: generates deferenced data object

 -- Procecure: cdata& data
     to be documented: generates pointer data object

Other Procedures
================

 -- Procedure: ctype-equal?
     to be documented This procedure is not dependent on ‘*arch*’.

Guile FFI Support
=================

 -- Procedure: ctype->ffi-type type
     Convert a _ctype_ to the (integer) code for the associated FFI
     type.

References
==========

  1. Guile Manual: <https://www.gnu.org/software/guile/manual>
  2. Scheme Bytestructures:
     <https://github.com/TaylanUB/scheme-bytestructures>