Re: pymacs

[François Pinard]

[Pavel Janík]

> when reading through the issues Gerd Moellman left us in after-21.1.xmail
> file, I have also found out your e-mail about pymacs.  Can you please
> write something about it again to emacs-devel so more people will look
> at it and it could be finally integrated into development sources?
> It would be really valuable addition to GNU Emacs.

Hello, Pavel, and other Emacs developers.

I just stumbled on the full `emacs-devel' address in another mail, as being
address@hidden', so I'm Cc:ing there.  I'm not on this mailing list,
so please Cc: me as needed, if you feel I should follow some conversation.

A few weeks ago, I sent a letter about Pymacs to Richard Stallman, who told
me that too many things were moving for him to have enough time to consider
it, and I promised to not burden him anymore on this topic.  I hope I am
not seriously breaking my promise by following up on your letter. :-)

Here is the body of the announcement letter for Pymacs 0.15:

---------------------------------------------------------------------->
Hi!  A new release of Pymacs is available as:

    http://www.iro.umontreal.ca/~pinard/pymacs/pymacs.tar.gz

Pymacs allows Emacs users to extend Emacs using Python, where they might have
traditionally used Emacs LISP.  Pymacs runs on systems having sub-processes.

One bug in the `rebox.py' example was reported the day 0.14 was published.
I waited a few days in case other bugs would be reported, but none shown up.
So, if you have 0.14, there is no need to switch to 0.15, unless you have
interest in the Pymacs example for itself.  I know some users do :-).
----------------------------------------------------------------------<

Here is the contents of the letter I sent Richard.

---------------------------------------------------------------------->
The main difficulty I see with integrating Pymacs in standard Emacs is that
there is an Emacs LISP part and a Python part, which need to be at the same
protocol level.  The protocol did not change in months, but we never know.
We might be running after difficulties if we have Emacs install the Emacs
LISP part, and hope to see the Python distribution asynchronously install
the Python part.  On that precise point, it might be better to have Emacs
install all parts automatically if Python is detected, or at least, offer
a recipe and necessary files for the user to install the Python part herself.

Pymacs is made up of `pymacs.el', `pymacs.py' and `pymacs-services'.

Emacs LISP installation of a `pymacs.el' file does not cause in itself
any new kind of problems, this has been done for a lot of other packages,
so I'll skip discussing it.

As for `pymacs-services', this is a rather small Python script, very
similar to any shell script, it should not be a problem either, and could be
installed along with other scripts not meant to be directly called by a user,
but by Emacs itself.  We might want to substitute one or two fields in the
script at installation time.  The first substitution might be the location
of the Python interpreter on the she-bang line, yet we could back out and
just use what people often write: `#!/usr/bin/env python', forcing Python
to be looked for along the search path.  The second substitution might be
the list of directories when Python extensions to Emacs are to be kept,
yet we could back out and just let the user install her extensions along
the actual Python search path for importable modules.

But we could do better, foreseeing a time when Emacs might convey Python
extensions as well.  The same as we have `/usr/share/emacs/21.1/lisp/' to
hold standard LISP modules, we might see `/usr/share/emacs/21.1/python/'
for standard Python extensions.  This directory could be empty at first,
but its path should be burned into `pymacs-services', with `--prefix'
substituted appropriately at installation time.  Similar considerations
might apply to a `.../site-python' to parallel `.../site-lisp'.

Installing `pymacs.py' may be done in many ways, as there is both Automake
support for installing such things, as well as support for the Python side,
using something called Distutils, which became standard in Python 2.2.
Emacs does not use Automake.  Using Distutils is rather easy, one writes
a smallish `setup.py' specification, and installation to standard Python
places is launched through calling `python setup.py install'.  Currently,
Pymacs uses its own bit-bigger `setup.py', not based on Distutils.  Pymacs
does not use C extensions nor need shared libraries, we have a very simple
case here.  The only needed thing is to produce `.pyc' file from the `.py'
file using the already installed Python interpreter, this may be done by an
all in-line Makefile rule, being a bit long, or else, through using a Python
script of two or three lines.  The usual way to do this is to install the
`.py' file first, then compile it right from and into the installed place.

Since `pymacs.py' is only needed by `pymacs-services' (for its server part)
and by Python-written extensions to Emacs (for its client part), the Python
search path might be fully under Emacs control in all cases, so it does not
need to be installed in standard Python locations.  I think it might better
be kept in the Emacs installed hierarchy, run-time extended by the user as
she sees fit: there is a LISP `pymacs-load-path' variable to that effect.
Directories in that variable get transmitted as extra arguments to the
`pymacs-services' script, which should prepend these to its built-in list.

So, you see, integrating Pymacs in Emacs is merely deciding how to do it.
Another decision remains, about how to make the documentation available.
Currently, this is a single Allout format file, maybe not worth a separate
manuel, maybe just a chapter or a section in an already existing manual...

For how to split the work, I suggest that you take care of taking proper
decisions, as well as modifying the Emacs distribution's Makefiles and such,
as needed.  I would rather not dive into the Emacs distribution proper.
My part could be providing Pymacs itself, modified according to the decisions
you take about how to integrate it, and maybe reformatting the documentation
as a Texinfo chapter or section, of course if you feel useful that I do this.

P.S. - There is also a patch by Gerd, implementing `post-gc-hook'.
That hook exists already in XEmacs.  This patch is not strictly necessary,
but it would help Pymacs at reclaiming Python memory a bit more timely.
When that variable does not exist, Pymacs uses a timer to clean up memory
at regular intervals.  Tell me if that patch was lost along the way of
Emacs getting back to you, I'll send the copy I saved and pre-tested, here.
----------------------------------------------------------------------<

-- 
François Pinard   http://www.iro.umontreal.ca/~pinard