Re: Fix for slow process output processing (please test).

[David Kastrup]

address@hidden (Kim F. Storm) writes:

> address@hidden (David Kastrup) writes:
> 
> > > David Kastrup and others have reported very slow processing of process
> > > output, particularly with the Linux kernel's scheduler giving emacs
> > > too much attention while starving the processing producing the output.
> > > 
> > > Below is a patch which introduces a small delay on reading output from
> > > such processes, without using global delays in processing other events
> > > (this is done by temporarily removing the process' file descriptor from
> > > the call to select and use a short timeout on the select instead.
> > 
> > I have glanced over that patch.  There is one problem I see with it:
> > if indeed the generating process is able to produce lots of data with
> > a single write call, Emacs will be throttling its input without a
> > purpose.
> 
> With the patch, emacs only throttles if it reads less than 1024 bytes
> at a time from the process -- i.e. if the process is unable to produce
> data fast enough to fill emacs's read buffer.
> 
> So if you have a process which manages to write a lot of data in one
> write, emacs should happily read that data with no throttling.

And if we have a process that writes in small chunks but with very
good CPU utilization (something like dd if=/dev/zero obs=1), then we
will alternate between reading 8192 (or whatever the pipe size is) and
1 byte.  Of course vastly preferable to alternating between reading 1
byte and 1 byte.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum