Re: Dynamic loading progress

[Stephen J. Turnbull]

Philipp Stephani writes:

 > > using various unwind-protect constructs.
 > 
 > It is not possible to write such constructs in general. Stack
 > frames that don't originate from C programs simply can't be part of
 > a non-local jump.  unwind-protect doesn't help at all because it
 > doesn't stop non-local jumps.

Er, what does "stop non-local jumps"?  What Emacs can do is catch its
own non-local jumps so as not to orphan non-Emacs stack frames.  It
can't catch (or prevent!) anybody else's.

 > > And I'm sure that there's non-robust code in libraries out there
 > > that will be totally horked on a non-local exit.  But AFAICS you
 > > have the same problem there if you link statically or at Emacs
 > > initialization -- that code just shouldn't be linked into Emacs
 > > period.

 > Such a decision would massively reduce the usefulness of
 > modules.

I don't see how.  Please explain which use cases you have in mind.

 > It would prevent dynamic modules in any language other
 > than C,

Why?  There's a long history of embedding other languages in Emacs
(CCL, perlmacs, pymacs).  Runtime optional loading aka dynamic modules
involves problems new to GNU Emacs, but the longjmp problem is not
new.  It just wasn't a problem in practice because those called
languages don't provide a callback interface (as implemented in those
projects).  They were used purely as subroutines to Emacs: call in,
get result or error, forget until next time.  No longjmps to Emacs out
of the embedded interpreter.

 > and even for C modules implementors would need to be careful not to
 > write longjmp-hosting code.  This is indeed unacceptable.

Heh.  The GPL and RMS's historic no-dynamic-modules policy are each
far more restrictive of code you might want to link with Emacs, yet
you accept them.

Politics aside, I wish you luck (and backups, early and often) when
using code that longjmps past Emacs stack frames, such as any embedded
language that implements its own exception handling.  You'll need it.

But that's not what is at issue here.  What is at issue is how to
handle the case where the module calls into foreign code and provides
a callback that calls Lisp, and thus might do

  [main Lisp] -> [module] -> [callback Lisp] --Fsignal--> [main Lisp]

leaving the module in an inconsistent state because its stack has been
blown away.  As far as I can see, this is a very rare case, limited to
(eg) embedding another full-blown interpreter in Emacs.  Modules like
libz, libcanna, and libpq (in XEmacs) simply don't provide callbacks,
so this scenario is impossible.  OTOH, somebody who really wants to
provide more access to Python internals than `comint' does is not
going to need or want Fcondition_case_for_dummies.

The way I propose to handle the issue is by providing a sample wrapper
based on condition-case for Lisp callbacks that handles longjmps out
of Fsignal (which I hope is the only way to longjmp in Emacs).  Thus:

                                    +-----------Fsignal-----------+
                                    |                             |
                                    V                             |
  [main Lisp] -> [module] -> [condition-case] -> [callback Lisp] -+
       A           |  A            | A                            |
       |           |  |            | |                            |
       +--return---+  +------------+ +-------normal return--------+

protecting the module.

I don't see what else can possibly work, since if you set a condition
case in the API calling into the module (ie, in [main Lisp]), you've
already hosed the called code by the time you handle it.  I don't see
how a *generic* (and mandatory, per Daniel's proposal) wrapper can be
provided, since the callback API will be defined by the called code,
not by Emacs.

The sample wrapper is likely to be quite generic, however, as many
typical callbacks are going to reduce to Fapply(lisp_code, lisp_data).
But it can't be fully generic.  Some callback APIs are going to supply
module-defined data structures and perhaps a different number of
arguments, and in those cases you'll need to write your own callback
wrapper that marshalls the arguments into Lisp.  Finally, you may
anticipate that certain kinds of errors will be common in the Lisp
code called, and you would like to handle those in your own way.  A
generic wrapper provided by the module API is either going to provide
only a generic "you are hosed" error return, or it's going to have to
reimplement condition-case.  Why not just use Fcondition_case itself?

And, of course, some users may prefer to write their own
condition-case in Lisp.

Bottom line on a mandatory API enforcing a condition-case: Mostly
YAGNI, most of the rest YCUIWYNI[1].

Footnotes: 
[1]  You Can't Use It Where You Need It.