Re: The event handling thread (was: Threading IO-bound functions)

[Eli Zaretskii]

> From: John Wiegley <address@hidden>
> Date: Mon, 19 Dec 2016 09:43:30 -0800
> Cc: Eli Zaretskii <address@hidden>, emacs-devel <address@hidden>
> 
> EM> Now, my thoughts on this is that keyboard entry is an inherently
> EM> single-threaded operation *from the user's point of view* and that the
> EM> Emacs platform should enforce this. Thus, keyboard input should only be
> EM> allowed from the main thread.
> 
> I too had thought keyboard events would be managed by an IO queueing thread,
> and dispatched to other threads when relevant (i.e., when the receipt of IO
> preempted another thread that was "awaiting input").

As I explained here, the de-facto "IO queueing thread" is the "primary
thread", the one started when Emacs starts.  This is by virtue of the
current design, whereby only one thread is ever allowed to wait on
each handle through which Emacs communicates with the external world;
that includes the keyboard handle.

This works by marking each handle with the thread that waits for it.
When a thread calls one of the APIs that need input from external
sources, such as accept-process-output, or when it is about to become
idle, the code in wait_reading_process_output generates the descriptor
sets to be passed to 'pselect', and excludes from the set any
descriptors that are already marked with some other thread waiting on
them.

Since the primary thread is the only one at session genesis, it finds
the descriptor for the keyboard input free and "up for grabs", so it
marks the keyboard descriptor with its thread ID when it calls
'pselect'.  Other threads can only run when the primary thread calls
'pselect', so they all find the keyboard descriptor already "owned" by
the primary thread.

That "ownership" will only be lifted when the primary thread exits
'pselect', acquires the global lock, and starts running again, at
which time all the other threads are either in their 'pselect' call or
wait for the global lock to become unlocked.  The primary thread will
run until it again enters wait_reading_process_output, at which time
it will find the keyboard descriptor either owned by itself or
unowned, and will mark it as owned by itself.  So the other threads
will again see it as owned by the primary thread.

This pattern will repeat itself forever, so I see no way for any
non-primary thread to grab the keyboard descriptor and wait on it,
unless we add something that would provide a means for them to do so,
or until the primary thread runs some Lisp that calls thread-yield or
some other synchronization primitive.

To see the above machinery in action, try this:

  (defun infloop ()
    (with-temp-buffer (while t (insert "foo"))))

  (make-thread #'infloop "thread-loop")

As soon as you start the looping thread, any keyboard input, like M-x
or cursor motion commands, doesn't have any effect, until you type C-g
(which causes the looping thread to exit), because the looping thread
never yields.

> That is, I never thought event handling would be entirely cooperative, since
> the user invoking a command should suspend what Emacs is doing, unless what
> Emacs was doing was awaiting input.

I'm not sure I understand what you are saying here.  Remember: only
one thread runs Lisp at any given time, so all the other threads are
already "suspended" when the user input arrives at whatever thread is
waiting on the keyboard descriptor.  If the current thread that runs
when the user types is not waiting on the keyboard descriptor, it will
never know the user typed something, and will continue running until
it yields or calls one of the "yielding" APIs, which end up in
wait_reading_process_output.  The thread that did wait on the keyboard
descriptor will exit its 'pselect' call, try to acquire the global
lock, and wait there for the running thread to yield.

> Since there's some confusion on this point -- and maybe what I've stated above
> doesn't make sense, since it could impose concurrency in places we've not
> thought about yet -- I'd like us to discuss this a bit more.

Well, let's discuss ;-)