Re: Dynamic loading progress

[Aurélien Aptel]

On Sun, Oct 4, 2015 at 6:30 AM, Tom Tromey <address@hidden> wrote:
> Tonight I tried to use the module code to write an FFI module using
> libffi.  I found some holes that make this difficult to do.

Cool :)

> It would be great to be able to make new types from C.  That would let
> me wrap up various C objects I'd like to return opaquely to Lisp.
> Wrapping pointers would be good enough (for now I am using bit vectors)
> but having more control would be much better.
>
> Overkill here is better.  You'll want to extend this vtable over time,
> for example to add a GC mark function.  I don't particularly care if the
> vtable is exposed or private.
>
> Of course, if you can add types from C, why not from Lisp?  But I guess
> I could write a module to allow that :)

We have to be careful with this vtable thing. I don't want to end up
with C++-level complexity with operator overloading, copy/move
constructor, etc. To store pointers you can currently use bit-vectors
as you did or also have a table that maps integers to pointers on the
module side. For example, a module could return new "types" using a
cons ('my-type . ID) where ID is an integer index into a pointer table
handled by the module author. It's more work for module writers but it
works and keeps the complexity of the module interface low.

The only thing we cannot do rigth now is finalizers. But looking
through the commits I saw that Daniel added support for "finalizer" in
emacs core but I haven't looked into what it actually does. Maybe we
can re-use this feature instead of doing it again basically.

> * I found the emacs_runtime / emacs_env split strange.  I don't know
>   what the purpose of this is.  Also there doesn't seem to be a way to

Daniel's design is modeled after the JNI spec where the env must be
used from the thread that created the runtime the env came from, IIRC.
I have added a "private" module_id field in the env structure which is
used to index a module global refs. I was also thinking that you could
have for example a module that lets you write sub-modules in python,
each with a different module_id but the same runtime.

>   free an emacs_env, which I suppose makes sense in a way; though
>   shouldn't they simply be GC'd as well?

Nothing can be unloaded in elisp but I didn't made that assumption for
modules and I've indexed global refs by module so that you can clear
them per module if we ever implement that.

> * module-call never frees "args".

Fixed, thanks.

> * For error handling for now I ended up just calling Emacs stuff
>   directly.

Ok.

If you have a public repo somewhere I'd be interested to have a look
at your experiements.