Re: Emacs deadlocks on sub-process reads and writes

[speech . free]

I can now verify that by locking out the blocking write we no longer have the 
deadlock. So this would appear to be an Emacs bug or feature. I am using the 
Emacs 24.3.1 (i386-mingw-nt6.2.9200) of 2013-03-17 on MARVIN binary downloaded 
from Gnu site.

On Saturday, October 18, 2014 6:48:43 PM UTC-7, speec...@gmail.com wrote:
> I am debugging a Clang based code completer (irony-mode). Unfortunately the 
> transfer between Emacs and the Clang sub-process is slow on Windows. We are 
> experiencing frequent freeze-ups due to what looks like a deadlock. The 
> deadlock happens under the following conditions:
> 
> 
> 
> 1. Emacs reading a large transfer from the sub-process (2+MB, around 2 
> seconds to finish on Windows after setting w32-pipe-read-delay to 0);
> 
> 
> 
> 2. While 1. is ongoing Emacs side send a new request to the sub-process. The 
> request is regular source file sized -- it includes the buffer content being 
> edited -- sufficient to cause a block on the write but nothing too big.
> 
> 
> 
> When these two conditions are met, Emacs UI reliably locks up until the 
> sub-process is killed. The error message complaining about writing to a dead 
> process after killing the sub-process shows that the Emacs was deadlocked 
> waiting for the write to finish. The process buffer is half filled showing 
> that the process filter function was no longer called. The write to the 
> sub-process, which blocks the UI, can't proceed as it exceeds the size of the 
> small write buffer. The sub-process will not read on its side until it 
> finishes the transfer to Emacs. But for some reason Emacs also stopped 
> reading from the pipe.
> 
> 
> 
> After poking around the Emacs code base a little bit it looks to me that when 
> you call process-send-* in lisp Emacs eventually calls send_process in 
> process.c. If the send can't complete it calls wait_reading_process_output, 
> which calls read_and_dispose_of_process_output, which if there is new text 
> received, calls the lisp process-filter. The logic in 
> wait_reading_process_output is too complicated for me to understand if 
> somehow it may decide not to call read_process_output, causing a deadlock. So 
> I am asking for advice here.
> 
> 
> 
> A previous related question asked about transfer speed on Windows 
> https://groups.google.com/forum/#!topic/gnu.emacs.help/Y0MYAz8WZUU I believe 
> the slowness is the reason why this manifests itself on Windows but it may 
> also appear on Linux under the right conditions. For a very long thread of 
> discussion on this issue see https://github.com/Sarcasm/irony-mode/issues/131
> 
> 
> 
> Please feel free to forward to any Emacs list that may be relevant to solving 
> this issue.