Re: Multithreading, again and again

[Stefan Monnier]

> 0. What we have, and where we go.
>    What's the status of git://gitorious.org/emacs-mt/emacs-mt.git and
>    http://bzr.savannah.gnu.org/r/emacs/concurrency? Whether there are
>    plans for further development?

I'd like to merge this concurrency branch into Emacs-25, yes.

> 1. To thread or not to thread?
>    Is there a consensus whether it's worth doing at all? If I don't miss
>    something important from previous discussions (referenced from
>    http://www.emacswiki.org/emacs/NoThreading, what a title), everyone
>    agrees that some kind of threading is useful - but maybe someone has
>    the votum separatum?

I don't think there is such a consensus, but better support for
concurrency is something I consider important.  It is currently much too
difficult to implement "asynchronous article fetch" and things like that.

> 2. Hide them all!
>    Should the threads be visible (may be created and managed) from Lisp,

Yes.

>    be implicitly threaded? Redisplay (one thread per frame)? Buffer/socket

Parallelising the redisplay would be great, yes, but that's a completely
orthogonal issue.  Making it work concurrently with Lisp could be
interesting as well, tho probably less important.

> 3. Too big, too global.
>    How to split/share/synchronize huge global state between threads?
>    What should be thread-local? (I feel (current-buffer) should be
>    global and max-lisp-eval-depth should be thread-local, but I suspect
>    that there are much more controversial cases). Is it reasonable to
>    have become-thread-local-on-write globals?

Part of the reason I'd like to see the concurrency branch merged is in
order to get more experience with such issues.  It's hard to know what
would work and what wouldn't.  Especially interactions between
dynamic, thread-local, and buffer-local bindings are very tricky.

>    What synchronization primitives should be implemented? Atomic
>    variables?  Mutexes/semaphores? Barriers? Higher-level
>    not-so-primitives like thread pools or multithreaded queues? Should
>    some basic operations, like setq, be always atomic? Is it
>    reasonable/feasible/useful to have special form like (with-atomic
>    BODY...), where all BODY forms are evaluated atomically with
>    respect to all other threads?

I think the only realistic choice for now is `yield'.  The second best
would be transactions like `with-atomic'.

I also agree that moving towards something like Guile might be a longer
term option.  But it's not clear whether that's ever going to happen.


        Stefan