Re: [Bug-apl] Request for feature: Back-channel interaction

[Juergen Sauermann]

Hello Elias,

1. Lets say your function is EE and you call it with a function number as axis argument like you did. (I believe
this is a good convention). Then

A EE[x] B in APL calls eval_AXB() in the shared library,
EE[x] B in APL calls eval_XB() in the shared library,
and so on.

There are 12 such eval functions in total, but the native user defined APL operators require a very deep
understanding of the GNU APL internals (if user-defiined functions are passed as operator arguments).
So assume only eval_AXB() and eval_XB() are used.

The arguments (A, X, and B for eval AXB() and X, and B for eval_XB are of type Value_P which are smart pointers
to class Value. Don't touch (delete etc.) them but rather use them like const Value *. So you can use all const and static
functions defined in Value.hh.

The first thing is to check the incoming values, you can use is_empty(), is_skalar() and
friends in Value.hh. get_shape() returns the shape of the values and so on.

If something is wrong you can RANK_ERROR, LENGTH_ERROR, ... which are macros that terminate execution of the
function and return the according APL error to the interpreter.

At some point you want to access the ravel of the value with get_ravel(n) which is (,B)[n - ⎕IO]. We are in C++ so indexes
start with 0. get_ravel() returns a Cell which is either an integer, a float, a complex number, a character, or another (nested) APL value.
The different Cell types are derived from the same base class Cell (see cell.hh) that also has access functions to them (get_int_val(),
get_real_val(), ...)

If your function has one or two arguments then use eval_XB() or eval_AXB(). If it has more then you can use nested APL values,
see fprintf() in file_io (to be committed).

Next you do something specific to your functions and then create a result of class Token. This generally goes in 5 steps:

a. create the shape for the result Value

b. Value_P Z(shape, LOC);   // create a value with proper shape but undefined ravel.

c. set up ravel elements with placement new on either &Z->get_ravel() if the ravel is initialised in random order, or,
   more likely, with Z->next_ravel() and Z->done() if the ravel is initialised in sequence. For example,
   new (Z->next_ravel()) IntCell(42) sets the next ravel element to integer 42.

d. call Z->check_value() unless you are sure that all elements of the ravel are initialised (you are not sure!).

e. return Token(TOK_APL_VALUE1, Z);

For common cases (skalars, vectors, strings) there are constructors in Value.hh that combine steps a and b, like:

b1. Value_P Z(new Value(UCS_string, LOC));

The #include files in native/template.hh are the ones containing the above functions.

It is essential that the Value_P object is created like described; don't take shortcuts on that! They would break the
new smart pointer-based memory scheme.

Note that your eval_XX() functions are now essentially new primitive APL functions. You can therefore look
for existing primitive APL functions in PrimitiveFunctions.cc to see the typical design pattern for such functions.

2. Generally speaking you can start a thread in your native function and that thread can then access all data
in the APL interpreter. None of the data structures in the interpreter is meant to be used by another tread so
it could happen that they disappear without your thread noticing that. It can also happen that one of the functions
you call throws an exception and returns without your lock being released (and then dead-locking the interpreter).

Not good. What I would propose instead is a simple variable set by the interpreter that tells if the interpreter expects
input (and is then guaranteed to not change its data structures until new input is received. Since you control the input
in emacs mode, when you see the variable set and send no data to the interpreter then you are safe.

3. Yes. I believe you need to sign a few papers (see chapter 6 of http://www.gnu.org/prep/maintain/maintain.html) then.

4. right now the .so files are installed in a subdirectory apl of /usr/local/lib or /usr/lib. This is under the control of
autoconf/automake and I believe the path to the parent directory of that subdirectory apl is known by the dynamic
linker by some magic that I do not fully understand. I have also tried to not install the .a and .la files, but have not
succeeded yet. Maybe a relative apl/ee,so works with ⎕FX - havent tried yet. If you have ideas - please let me know.

Best Regards and a happy new year,

Jürgen Sauermann


On 01/02/2014 11:47 AM, Elias Mårtenson wrote:

And I have started work on a native module to provide direct access to some internals of GNU APL using a simple protocol.

The basics works right now, but I have a few questions:

1. I have a command (EE[1]) that starts the server. I would like to pass in an array of arguments (like port number, for example). What is the correct way to read these values?
2. Right now, I just call functions on the workspace directly, and since I do this from a separate thread, it's guaranteed that this will give us race conditions and cache coherency issues. Until a "proper" threading solution is in place, how about simply having a single lock that is held while anything is touched? (in your code, that lock would be held at all times except for when you're waiting for input by the user)
3. Could this module be shipped as part of the GNU APL distribution? That way it would be compiled as part of the normal installation and there would be no problems installing the Emacs support.
4. I'm building and installing GNU APL into a separate directory (i.e. not /usr/local) and I find that I have to specify the full path to the .so file when I load it with ⎕FX. Is there a correct way to load a library relative the GNU APL installation directory?

Regards,

Elias