Re: Are there plans for a multi-threaded Emacs?

[David Kastrup]

Ted Zlatanov <address@hidden> writes:

> On 04 Dec 2003, address@hidden wrote:

[...]

> >> (Xeons).  I don't have benchmarks, sorry, but at least on Solaris
> >> the performance of a machine increases by at least 80% with each
> >> additional SPARC processor.
> > 
> > Such naive sweeping claims would tend to ruin your credibility,
> > I'm afraid.
> 
> OK, here's some Novell benchmarks that show significant increases in
> performance with additional processors in Solaris in a particular
> application (LDAP server):
> 
> http://www.novell.com/info/collateral/docs/4621167.01/4621167.html
> 
> I work with Solaris for a living, so I feel justified in making that
> claim based on what I know - years of experience, Sun's published
> benchmarks, vendors' benchmarks.

Sun would hardly publish benchmarks centered around applications that
don't gain from multithreading/multitasking.

We are not talking about an LDAP server, but Emacs.  Ironically,
using applications like preview-latex or in general comint-mode,
Emacs will show _vast_ speed improvements (decidedly more than 100%)
under multiprocessor machines for most operating systems, and the
reason is a scheduling anomaly for almost all operating systems on
single processor machines that keeps pipes from getting more than
minimal fill levels: as soon as there is something in the pipe, the
pipe filling process gets starved from CPU, and so even a tight loop
of
  for (;;) write(1,"\n",1);
will never manage to get more than a single character processed by
Emacs' filter functions.  With a multiple CPU box, this does not
happen as the pipe fills even while Emacs is processing the last bit.

I have gotten real time speed improvements even by starting a
completely useless busy loop that would get the scheduler involved
more often than when Emacs and the comint-run application where just
passing the CPU among each other.

So here you have a situation where a two-processor machine will
increase speed by decidedly more than 100%, and that even without
Emacs itself being multithreading.  I'd be careful about just what
conclusions to draw from this.

Similar from Solaris benchmarks.

> Note I did not say that an *application's* performance increases by
> at least 80%.  The machine's performance, given processes or threads
> within them that can be effectively run on separate processors by
> the OS scheduler, is what increases.  This is generally understood
> but perhaps my language was not clear enough.
> 
> Specifically in the context of Emacs, I do not know the internals
> well enough to judge that.  I hope I can help with the work in any
> way I can, no matter what the eventual "multithreading or not"
> decision is.

My personal opinion is that the Lisp machine would need to get
reorganized significantly to even get a modicum of advantage.  And
that implies, among others, a clean separation between global entities
and locally bound entities that I don't see usefully implemented
without lexical binding.  I may sound as though I was riding a hobby
horse here, but I am not involved in that work, and while I consider
it an issue of architectural cleanliness, I have my doubts about the
feasability of change here.  The longer it is postponed, the more
painful a change will get.  And a change in there would bear quite
more consequences to existing Lisp files than, say, the current
garbage collector and memory layout changes.

One may be able to duct tape some measure of multithreading even on
top of pervasive dynamic binding, but to be honest, I fear the
outcome.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum