Re: Concurrency via isolated process/thread

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: luangruo@yahoo.com, emacs-devel@gnu.org
> Date: Sat, 08 Jul 2023 08:16:39 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> So, it can be something like copy-on-read - the child thread will copy
> the necessary values from parent thread as running Elisp code needs
> them.
> 
> By default, such copy will not be synchronized with the parent Emacs
> thread, so that we do not need to worry about parent thread changing the
> values asynchronously. 

When does such synchronization happen, though?

Imagine some long-running thread which does some periodical
processing: when and how will such a thread synchronize with the
actual state of the variables it needs?  How will this look from the
POV of Lisp code running in this thread?

(This proposal also means significant implementation problems.)

> Manually, it may also be allowed to create "remote" values that will
> query the other thread: thread 1 will hold (foo = '(1 2 3)) and thread 2
> will hold (foo = #<remote foo>). Then, thread 2 requesting to read/write
> value will query thread 1.

A thread running Lisp cannot be queried, unless it somehow "listens"
to such queries.  This isn't magic, it has to be implemented, and how
do you envision to implement it?  You envision some kind of scheduler
as part of Emacs? something else?

> The range of variables that can be made remote should be minimized and
> possibly also restricted to a safe subset of variables.

I very much doubt such a "safe subset" exists that can support useful
Lisp programs.

> > ... If I am the Lisp programmer writing code for such
> > a thread, how can I know what is and what isn't allowed?
> 
> Everything is allowed, but global state changes will not necessarily
> propagate to the parent Emacs thread. They will be confined to that
> child async thread.

So it could be that a thread decides, for example, that some buffer B
is the return value of its function, whereas buffer B no longer exists
because it was killed?  How's that useful?

> > ...And what
> > happens if I do something that is not allowed?  And finally, does it
> > mean we cannot run existing Lisp programs in such threads, but must
> > program for them from scratch?
> 
> Non-trivial programs that need to modify the global Emacs state will
> have to be written specially.

So you basically agree that to have useful enough multi-threading,
most of Emacs will have to be redesigned and rewritten?

> But programs that only return a value will work as usual, without a
> need to modify them.

How many programs like that exist and are useful?

> The idea is similar to https://github.com/jwiegley/emacs-async (or to
> `org-export-async-start'), but with more tight communication between
> Emacs processes. The original emacs-async requires a new Emacs instance
> that will also need to re-load all the necessary packages manually from
> startup.

If we will have the same disadvantages and restrictions as in
emacs-async, then why bother? emacs-async already exists.