Re: What's the problem? (Was: Are there plans for a multi-threaded Emacs?)

[David Kastrup]

Ted Zlatanov <address@hidden> writes:

> On 08 Dec 2003, address@hidden wrote:
> 
> > What's the biggest problem that threads are intended to solve?
> 
> Note I don't claim these can't be rewritten in a concurrent fashion.
> I simply gave examples that could stand to be improved.  The
> majority of examples are in Gnus, because that's the Elisp
> application I know best.

Have you already done the obvious thing of setting gnus-asynchronous
to a non-nil value?  That's the least one would expect from somebody
that wants parallelism that badly.

> - auto-save on a slow file system or through Tramp

So that if autosave starts saving a file and is half through when I
decide that the first half of the file is junk I won't need and delete
it, Emacs finds itself having saved more than enough material and
stops writing, never saving the remaining _important_ part of the
file?

You can't just throw in parallelism and hope that things will work
out somehow.

> - Gnus mail retrieval, summary thread building, registry lookups

Most of those can be controlled with gnus-asynchronous and
subordinate variables.

> - independent hashtable lookups and calculations in parallel would
>   be a very nice improvement in themselves,

Why?  What time-critical code performs them frequently?

> > People are writing concurrent programs in Elisp today. Most
> > programs interacting with external processes and sockets do so
> > without blocking Emacs. Can't we all just do the same?
> 
> There seems to be interest in a real multithreading solution, be it
> cooperative or preemptive, or an event-driven API.

The interest I have heard voiced so far has not been overly correlated
with intimate technical knowledge about the details involved and the
actual differences this would make for everyday work.

> If it appears from this thread that I have a particular interest in
> a radical rewrite of Emacs, that is not true.  It just seems to me,
> because of the tight internal coupling of Emacs functions and
> variables, that lightweight threads would be a good choice for the
> internal implementation of the API because they have less baggage
> than process sentinels and share data more easily.

And sharing data more easily is exactly what one can't afford in a
language with dynamic binding, since you get no closures whatsoever.
If I have something like

(let ((x (+ y 2)))
  do something)

then I can't afford to have some other thread change the meaning of x
in his own (let ((x ... construct.  If you want to afford a separate
binding stack for every thread, this also means that every _read_
_access_ to a symbol must run via the thread's binding stack instead
of just using the stack whenever the a binding _changes_.

And it is not only local variables that get temporarily bound: there
are common constructs like
(let ((process-connection-type nil)) [...])
where global variables get some value for some short duration.

> I also mentioned that today's hardware and OS support for it
> generally means that either multiple processors or the illusion of
> them is available to applications that are willing to make use of
> those capabilities.

Applications don't have a will of their own, even though it sometimes
may appear so.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum