Re: Any idea about what makes Emacs slow reading on pipes?

[David Kastrup]

"Jan D." <address@hidden> writes:

> >
> > The following exhibits abysmal speed within Emacs:
> >
> > (let (process-connection-type) (switch-to-buffer (generate-new-buffer
> > "*test*"))(erase-buffer)(start-process "test" (current-buffer) "sh"
> > "-c" "hexdump -v /dev/zero|dd bs=1 count=100k")(erase-buffer))
> 
> ...
> 
> > The system I see this in is
> > GNU Emacs 21.3.50.3 (i686-pc-linux-gnu)
> > a RedHat 9 system, but I have seen this in practically every Linux
> > system up to now.  Is this a Linux-specific problem, or do others see
> > this as well?
> 
> I could not reproduce this on a slackware (version unknown, but libc 6)
> or a Mandrake 9.0 system.  Maybe running under strace can give some
> indication of the system call that takes time (if that is the issue)?

Here is a more elaborate test file

testio.el
Description: Check filter function strangeness

If I run this and print out test-pattern afterwards, I get

test-pattern

((485 0 11) (1024 0 11) (1024 0 11) (1 0 11) (1 0 11) (1 0 11) (1 0
11) (1 0 11) (1 0 11) (1 0 11) (1 0 11) (1 0 11) (1 0 11) (1 0 11) (1
0 11) (1 0 11) (1 0 11) (1 0 11) (1 0 11) (1 0 11) (1 0 11) (1 0 11)
(1 0 11) (1 0 11) (1 0 11) (1 0 11) (1 0 11) (1023 0 11) (1024 0 11)
(1024 0 11) (1024 0 11) (1 0 11) (1 0 11) (1 0 11) (1 0 11) (1 0 11)
(1 0 11) (1 0 11) (1 0 11) (1 0 10) (1 0 10) (1 0 10) (1 0 10) (1 0
10) (1 0 10) (1023 0 10) (1024 0 10) (1024 0 10) (1024 0 10) (1 0 10)
(1 0 10) (1 0 10) (1 0 10) (1 0 10) (1 0 10) (1 0 10) (1 0 10) (1 0
10) (1 0 10) (1 0 10) (1 0 10) (1 0 10) (1 0 10) (1 0 10) (1 0 10) (1
0 10) (1 0 10) (1 0 10) (1023 0 10) (1024 0 10) (1024 0 10) (1024 0
10) (1 0 10) (1 0 10) (1 0 10) (1 0 10) (1 0 10) (1 0 10) (1 0 10) (1
0 10) (1 0 10) (1 0 10) (1 0 10) (1 0 10) (1 0 10) (1 0 10) (1 0 10)
(1 0 10) (1 0 10) (1 0 10) (1 0 10) (1 0 10) (1 0 10) (1023 0 10)

[...]

(1 0 8) (1 0 8) (1 0 8) (1 0 8) (1 0 8) (1 0 8) (1 0 8) (1 0 8) (1 0
8) (1 0 8) (1 0 8) (1 0 8) (1023 0 8) (1024 0 8) (1024 0 8) (1024 0 8)
(1 0 8) (1 0 8) (1 0 8) (1 0 8) (1 0 8) (1 0 8) (1 0 8) (1 0 8) (1 0
8) (1 0 8) (1 0 8) (1 0 8) (1 0 8) (1 0 8) (1 0 8) (1 0 8) (1 0 7) (1
0 7) (1 0 7) (1 0 7) (1 0 7) (1 0 7) (1 0 7) (1 0 7) (1 0 7) (1 0 7)
(1023 0 7) (1024 0 7) (1024 0 7) (1024 0 7) (1 0 7) (1 0 7) (1 0 7) (1
0 7) (1 0 7) (1 0 7) (1 0 7) (1 0 7) (1 0 7) (1 0 7) (1 0 7) (1 0 7)
(1 0 7) (1 0 7) (1 0 7) (1 0 7) (1 0 7) (1 0 7) (1 0 7) (1 0 7) (1 0
7) (1 0 7) (1 0 7) (1 0 7) (1023 0 7) (1024 0 7) (1024 0 7) (1024 0 7)
(1 0 7) (1 0 7) (1 0 7) (1 0 7) (1 0 7) (1 0 7) (1 0 7) (1 0 7) (1 0
7) (1 0 7) (1 0 7) (1 0 7) (1 0 7) (1 0 7) (1 0 7) (1 0 7) (1 0 7) (1

[...]

0 6) (1 0 6) (1 0 6) (1 0 6) (1023 0 6) (1024 0 6) (1024 0 6) (1024 0
6) (1 0 6) (1 0 6) (1 0 6) (1 0 6) (1 0 6) (1 0 6) (1 0 6) (1 0 6) (1
0 6) (1 0 6) (1 0 6) (1 0 6) (1 0 6) (1 0 6) (1 0 6) (1 0 6) (1 0 6)
(1 0 6) (1 0 6) (1 0 6) (1 0 6) (1 0 6) (1 0 6) (1 0 6) (1 0 6) (1 0
6) (1 0 6) (1 0 6) (1 0 6) (1 0 6) (1 0 6) (1 0 6) (1 0 6) (1 0 6) (1
0 6) (1023 0 6) (1024 0 6) (1024 0 6) (1024 0 6) (1 0 6) (1 0 6) (1 0
6) (1 0 6) (1 0 6) (1 0 6) (1 0 6) (1 0 6) (1 0 6) (1 0 6) (1 0 6) (1

And from then on (actually backwards in time) only packets with single
characters, with perhaps 100 filter calls per second.

Which in itself is not too hot a speed, but gets awful in consequence
because the filter routine gets only called for single characters.

I don't know _what_ Emacs does do the pipe/pty that causes only single
characters to gather/be processed, with a sporadic 1024 packet getting
through from time to time, but I hate it.  Some setting must be on the
pipe that causes the writer to stall before further writes in many
cases until Emacs has read the next character.  And after some time
the pipe gets filled properly now and then, with single characters in
between.

I don't get it.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum