Re: Any idea about what makes Emacs slow reading on pipes?

[David Kastrup]

address@hidden (Kai Großjohann) writes:

> address@hidden (David Kastrup) writes:
> 
> > So what can we do about this?
> >
> > a) use only SMP systems
> > b) pester Linux developers to be less eager with context switches on
> >    select.  I am trying this course now, but it will of course take
> >    time to register and other OS might have similar problems
> > c) make a super-efficient path for process output arriving in tiny
> >    chunks, since this is what we will be force-fed most of the time.
> 
> I wonder if the invocation of the process filter function could be
> optimized.
> 
> Idea: the current situation might be this: when Emacs gets stuff from
> a process, it reads the data, stashes it somewhere in a buffer, then
> invokes the process filter function on the stashed-away stuff.
> 
> We might change it so that, after stashing the stuff in a buffer,
> Emacs looks again if more data is available from the same process.

The data will not become available unless the process gets the CPU
time to produce it, and we can't persuade the operating system to
give the process the CPU time unless we yield the CPU.  The solution,
as explained elsewhere, is to throttle the calls to the process
filter function and actively delay processing if we find ourselves
processing small packets at a high rate.

> That way, Emacs might continue reading stuff from the process until
> the process has nothing to produce.

Well, people always complain about Emacs being an operating system,
not an editor, but scheduling processes is not yet its responsibility.
I just propose that we help out the operating system in its scheduling
a bit...  since the bad consequences occur on our side of the
equation, we should do what we can reasonable do to avoid them even
where, strictly speaking, the OS is to blame for assuming that Emacs
will be happier with many small packets instead of a few large ones.

But the "slow start xterm" effect, where the performance of an xterm
with

od -v /dev/zero|dd obs=1

gets much faster if you start

while true;do :;done

in a separate term is very common on single-processor machines.  I
always found it curious and never thought about it further.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum