bug#22344: 25.0.50 (and 24.5.1, 24.4); Sending 4097 bytes to a subprocess hangs Emacs on Windows 8 and 10

[Eli Zaretskii]

> From: Clément Pit--Claudel <clement.pit@gmail.com>
> Date: Mon, 11 Jan 2016 00:42:01 -0500
> 
> On Windows, sending more that 4096 bytes at once to a subprocess will cause 
> Emacs to hang (freeze) indefinitely.

That's not true, at least not when phrased in such a general form.
See below.

> 1. Create a loop.cmd file in C:\. Put the following in it:
> 
> :loop
> goto loop
> 
> 2. Run the following snippet in emacs -Q:
> 
> (process-send-string
>  (start-process "bug" nil "c:/loop.cmd")
>  (make-string 4097 ?a))
> 
> 3. Emacs hangs.

That's expected, when your subprocess doesn't read its stdin.  Emacs
on MS-Windows uses pipes to communicate with subprocesses; a pipe on
Windows has a 4K buffer.  So as soon as Emacs writes more than 4096
bytes to the pipe, the pipe becomes full, and Emacs then waits for the
subprocess to read its end of the pipe, at which time Emacs will write
the rest of the stuff.  Since your batch file doesn't read its stdin,
Emacs waits forever.

If I modify your loop.cmd to have the following trivial command:

  @cat > foobar

then Emacs doesn't hang, it happily writes those 4097 bytes, and after
that, I see all of them in the 'foobar' file.  Moreover, I tried the
same test with 100 times as many bytes (i.e. 409700), and it still
worked as expected, with all of the bytes ending up in the file.

I also tried a more elaborate batch file that reads stdin in small
chunks, sleeping 1 sec between chunks.  That, too, didn't cause Emacs
to hang, although it took some time to run to completion, due to the
1-sec waits.

So the issue is more complicated than what you describe above, since I
believe the real-life use cases involved programs that do read their
stdin.

> I do not use Windows often, but I'll be happy to provide more information, if 
> I 
> can help. I did not compile my own Emacs; I used the one provided at 
> https://sourceforge.net/projects/emacsbinw64/ . The original bug report about 
> Flycheck, which did not identify process-send-string or process-send-region 
> as 
> the culprit, is here: https://github.com/flycheck/flycheck/issues/794 . I 
> have 
> included debug output below.

There could be some flaw in the logic that handles the overflowing
pipe, which your use cases somehow trigger.  But it's not a silly bug
like what your description suggests: large writes to subprocesses do
work, this was tested extensively back in Sep 2014 when bug#18420 was
fixed by making the pipes to subprocesses non-blocking.

Or maybe some other factor is at work in the real-life use cases.

So someone with access to the programs in question will have to step
with a debugger through the code involved in this (I can point to the
relevant parts of the Emacs C code), and find out what goes wrong
there, and why.  Alternatively, if a reproducible test case can be
posted with a simple program which reads its stdin and causes the
hang, I will try debugging it myself.  (By "simple program" I meant
something that won't involve installing lots of other packages,
ideally something as simple as a batch file calling 'cat' or a similar
program.)

Thanks.