emacs-devel
[Top][All Lists]
Advanced
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Dynamic loading progress

From: Aurélien Aptel
Subject: Re: Dynamic loading progress
Date: Thu, 9 Oct 2014 19:08:47 +0200 
Good news everyone!

As of my last commit (7a38270cd) dynamic loading is working pretty
well.

I've written a few toy modules in the "modules" folder. They can be
used as examples and cover various use-case.

- The `fmod' module simply expose the fmod(3) function from the math
  lib.

- The `elisp' module evaluates code written in Lisp from C.

- The `opaque' module show how any C structs/type can be embedded in a
  Lisp_Object.

- The `curl' module can actually be useful as it links against libcurl
  and lets you fetch any http url supported by curl. You can use the
  same curl handle for multiple request in order to reuse the
  connection when possible (faster!).

* Usage

Dynamic loading can be enabled/disabled via the configure
script. libltdl/ltdl.h is all you need to make it work. I've turned
the option on by default (--with-ltdl).

  # build emacs
  $ git clone https://github.com/aaptel/emacs-dynamic-module.git emacs-dyn
  $ cd emacs-dyn
  $ ./autogen.sh
  $ ./configure
  $ make

  # build all the modules
  $ for i in modules/*; do make -C "$i"; done

  # run emacs! (no init file)
  $ ./src/emacs -Q

Switch to *scratch* to test interactively the curl module:

  (add-to-list 'load-path "path/to/emacs/modules/curl")
  (require 'curl)
  => t

  ;; alternatively you can use
  ;;     (load "path/to/curl/curl.so")
  ;; or
  ;;     (load-module "path/to/curl/curl.so")

  (defvar c)
  (setq c (curl-make)) ;; allocate/init a curl handle
  => #<save-value <pointer 0x1cf56d0> <unused> <unused> <unused>>

  (curl-fetch-url c "http://example.com";) ;; make and store request
  => t

  (curl-content c)
  "<!doctype html>
  <html>
  <head>
      <title>Example Domain</title>
  ..."

  (curl-free c) ;; free the curl handle
  => t


I've just tested the build procedure above before posting and *of course*
for some reason a clean emacs build fail on something completely
unrelated. Even at my first commit where I didn't make any change yet.

  make[1]: *** No rule to make target 'ctags.c', needed by 'ctags'.  Stop.

I suppose it comes from my setup? Gahh.. Anyway...

* Implementation details and potential issues

** Loading

The actual loading is done in Fload_module in lread.c. It is called
from Fload when appropriate. See my previous emails.

When dynamic modules are enabled, the `load-suffixes' variable has the
additional .so suffix. This variable is used by `load' which in turn
calls openp() to do the actual search in `load-path'.

Adapting `load' to support modules made `require' and `provide' work
with no additional work.

Nothing is preventing a module from being loaded several times apart
from the `features' mechanism. This might be problematic.

** Documentation

I have updated several documentation related functions in order to
support multiple DOCFILE (each module has its own). See my previous
emails.

The xref on function documentation (go to definition) is currently
broken. I have not looked into it (issue #6).

I have not fixed the issue (#1) Stefan mentioned earlier about a
memory leak in the GC on the subr doc field because I *still* don't
get it :( Any help is welcomed.

** Opaque user types

Adding a new proper type is too much work for a module. You need to
provide a reader, a printer, add a bunch of enums, etc.

I've reused the Save_Value type to store arbitrary pointers. The
object hierarchy is as follows:

Lisp_Object > Lisp_Misc > Lisp_Save_Value > void* pointer

A major problem with this is that you can't make type predicates
e.g. `curl-handle-p'. A solution would be to make a new generic user
type composed of a user pointer and a cookie or some sort of ID we can
test. The space has to be large enough to limit collisions.

** Resource management

An allocated curl handle has to be explicitly freed with
`curl-free'. Could we extend the GC to provide some sort of destructor
mechanism? Just a thought.

Also, Lisp_Objects contained in a user type are not protected from the
GC. This can be a problem.

** Native type size

EMACS_INT_MAX is not always equal to INT_MAX depending on the
architecture. Libraries can often use special value to signal
something (e.g. (unsigned)-1 which is 0xffff..) which might not be fit
in a Lisp type. What can we do about it? Is it really a problem? Emacs
Lisp provides other way to signal errors and a module doesn't have to
expose low-level stuff like this.

** Packaging

Right now each module has its own Makefile and is built separately from
emacs. We need to think of a way to package modules for distribution
that is practical and takes into account what the module needs from
the Emacs build (which configure option should be enabled to build
module xxx?).

** Portability

The .so suffix is hardcoded but via libltdl loading should work on
most systems.
reply via email to
[Prev in Thread] Current Thread [Next in Thread]