Re: Concurrency via isolated process/thread

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: luangruo@yahoo.com, emacs-devel@gnu.org
> Date: Mon, 24 Jul 2023 14:24:32 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > So the goal is to eventually support true concurrency in Emacs?
> 
> Yes.

OK, thanks.

> > ... so these steps only make sense if the
> > concurrency is attainable via these measures?
> 
> Yes, except that the part about thread-local point and restriction may
> be more generally useful.

I suggest to forget that latter part: we have a "generally useful"
code already, so changing it without a very good reason is not wise,
from where I stand.

> > You are considering just the simplest scenario.  Usually, Lisp
> > programs in Emacs not only read text, they also write and delete it.
> > What if one thread makes changes to buffer text and another thread
> > then wants to access it using its state variables?  E.g., what do you
> > plan doing about the gap? will thread switch move the gap or
> > something? that alone is a performance killer.
> 
> AFAIK, reading buffer does not require moving the gap.

We've been through that: at least xml.c moves the gap to allow
external libraries access to buffer text as a single contiguous C
string.  This is a capability I wouldn't want to lose, because it
might come in handy in future developments.

> We only need to move the gap when buffer is changed or before copying
> text region from buffer to another buffer. Both such operations should
> be considered buffer modifications and must be blocking.
> 
> > To be a step in the general direction of achieving concurrency, we
> > need some plan that will at least in principle allow concurrent
> > editing.  Even if "concurrent" here means that only one thread can
> > write to a buffer at a time, we still need to see how this will work
> > when other threads are unblocked once the writer thread is done
> > writing.  Can you describe how this will work, assuming we keep the
> > current design of buffer with a gap?
> 
> The idea is the same as with what is already done for indirect buffers.
> Indirect buffer modification will affect buffer_text object in the
> base buffer (automatically - buffer_text object is shared). And they
> will also affect point position in the base buffer.
> 
> The point adjustment in the base buffer is done simply by storing point
> as a marker. We can do the same for thread-local point positions.

I still don't quite see how this will work.  Indirect buffers don't
introduce parallelism, and programs that modify indirect buffers
_know_ that the text of the base buffer will also be modified.  By
contrast, a thread that has been preempted won't know and won't expect
that.  It could, for example, keep buffer positions in simple
variables, not in markers; almost all Lisp programs do that, and use
markers only in very special situations.

In addition, on the C level, some code computes pointers to buffer
text via BYTE_POS_ADDR, and then uses the pointer as any C program
would.  If such a thread is suspended, and some other thread modifies
buffer text in the meantime, all those pointers will be invalid, and
we have bugs.  So it looks like, if we want to allow concurrent access
to buffers from several threads, we will have a lot of code rewriting
on our hands, and the rewritten code will be less efficient, because
it will have to always access buffer text via buffer positions and
macros like FETCH_BYTE and fetch_char_advance; access through char *
pointers will be lost forever.

So maybe we should take a step back and consider a restriction that
only one thread can access a buffer at any given time?  WDYT?