Re: Multithreading, again and again

[Stefan Monnier]

> yet.  I could do that if there is interest.  It is not yet at par with
> the old concurrency branch; in fact in ways it is very far behind (I
> didn't tackle bindings yet -- the code changed on the trunk and I'm not
> sure the old code was correct anyhow).

Indeed, the bindings code has changed on the trunk a bit (hopefully it
is a bit more clear now).  Also the lexical-binding code (which mostly
hasn't touched the "bindings" code) means that the performance of
thread-local let bindings is a bit less important (only a bit, tho,
since there are still many variables one wants to let-bind and that are
used by the C code in performance sensitive areas).

> One is DNS lookup.  This should be threaded internally, so that DNS
> delays don't hang Emacs.  This could actually be done today without
> requiring exposing threads to Emacs.

Yes, that's actually largely unrelated to "multithreading in Emacs", but
making it interruptible (and maybe even let process-filters and timers
run during this time) would be a good change.

I also recently noticed that sometimes my Gnus is unresponsive for
a while (waiting for a remote server, maybe in DNS, tho I think not) and
while C-g does work, other things don't, most importantly clipboard
requests get delayed so if my Gnus session happens to own the clipboard
then C-y in another Emacs session hangs until the remote server replies.

> The other is the multiple keyboard case.  I think it should be possible
> to have each 'emacsclient -t' run in a separate thread.

This OTOH is pretty much "the general case" since it needs to deal with
running Elisp in different threads concurrently (and with recursive
edits and minibuffers, we want to be able to run one terminal's command
while some other terminal still has active let-bindings).

> People have talked about making redisplay run in its own thread, but I
> don't know anything about redisplay and so I haven't looked at it.

That one is mostly a classic "make an existing C program concurrent by
adding locks where needed".

> My plan was to follow the Bordeaux threads API, more or less.

I'm afraid that using locks is not going to work well for Elisp.  I have
much higher expectations for approaches based on yield (i.e. approaches
which are inherently safe(r), with the main risk being lack of
concurrency rather than race-conditions and corruption).

Maybe we can even make yield work for true concurrency by treating it as
a kind of "end previous transaction and then start a new one".

> I think right now you can do things like:

>    (let ((some-binding))
>      (while (condition)
>        (sit-for 0)))

> ... and expect that some-binding will be visible to the process filters.

Yes, that's a code pattern that doesn't interact well with threads.
We'll need to adjust/fix those cases one by one, I think.


        Stefan