bug#13079: 24.3.50; Emacs cannot create subprocess

[Eli Zaretskii]

> Date: Wed, 5 Dec 2012 10:26:51 +0800
> From: Li Zhai <mrzhaili@gmail.com>
> Cc: 13079@debbugs.gnu.org
> 
> I mean, emacs display the messages before the loop ends. And all the
> processes are gone, emacs did not wait them to exit.

Emacs doesn't wait because you told it so, by using zero as the 5th
argument to call-process-region.  This is normal.

> I've tried two different version from
> http://ftp.gnu.org/gnu/emacs/windows/emacs-23.4-bin-i386.zip and
> http://alpha.gnu.org/gnu/emacs/windows/emacs-20120917-r110062-bin-i386.zip
> 
> Here is how to reproduce the bug:
> 
> 1. Open the emacs using ``emacs -Q''
> 
> 2. paste the code into *scratch* buffer:
> 
>   (progn
>    (dotimes (i 35)
>      (call-process-region (point-min) (point-min) "ddeclient" t 0 nil
>  "SUMATRA" "control")
>      (message "%d" i)))
> 
> the *ONLY* difference between the previous code is I increase the
> repeat times to call the function `call-process-region'. Here is 35
> times, you could try to increase the number lager if the bug not
> reproducing.
> 
> 3. Eval the code using C-x C-e
> 
> 4. You should see the emacs report an error:
> 
> Debugger entered--Lisp error: (file-error "Spawning child process"
> "resource temporarily unavailable")

This is not a bug, but a known limitation of Emacs on Windows.  The
way management of subprocesses is implemented on Windows, Emacs can
only have 32 subprocesses simultaneously at any given time (because it
watches two separate handles for each subprocess, and uses an API that
supports maximum of 64 handles).  So this is also normal.

If you use 32 instead of 35, the above error message will not appear.
But that won't solve the _real_ problem here, see below.

> 5. Quit the elisp debugger and execute the code again(press C-x C-e),
> this time emacs should report:
> 
> Debugger entered--Lisp error: (file-error "Opening output file"
> "permission denied" "c:/Users/iot003/Desktop/emacs-24.2.50/bin")
>   call-process-region(1 1 "ddeclient" t 0 nil "SUMATRA" "control")
> 
> 6. Quit the elisp debugger. This time, emacs cannot create any
> sub-process any more. Try to execute a shell command by press `M-!
> ls', you will find it out.
> 
> This problem occurred more often on my private build version. I guess
> the problem is the default stack size assigned by compiler is too
> small.

No, this has nothing to do with something fancy like stack size, it's
a much more mundane problem.  It's because we fail to create temporary
files needed by call-process-region and commands that use it.  Did you
look in your TEMP directory?  I bet you have gobs of emXYZZY files
there, exactly 42 of them per each time you tried this stuff.

The immediate reason for the mysterious "permission denied" error
(about a directory!) is that call-process-region was not checking
whether the call to 'mktemp' failed.  When that function fails, it
produces an empty "file name", and call-process-region was boldly
using that as a file name, eventually trying to open a directory,
which on Windows fails with EACCES.  This part was easy to fix:
starting with revision 110996 on the emacs-24 branch, Emacs now
produces a more intelligent error message in this case.

The harder part of this bug is the reason _why_ 'mktemp' fails.  Your
command launches asynchronous subprocesses, creating a temporary file
for each one of them where Emacs puts the contents of the region.  In
this case, call-process-region arranges for the temporary files to be
removed when the call to call-process returns, and then calls
call-process.  However, since this is an async subprocess,
call-process returns immediately, without waiting for the subprocess
to exit, and Emacs deletes the temporary file.  This "cleanup" uses
the non-portable (outside of Posix filesystems) trick of deleting a
file while it is still open and used by the child process.  On
Windows, this attempt to delete the file fails, and the file is left
behind.

The other part of this puzzle is that 'mktemp' as implemented by the
Windows runtime can only have up to 42 simultaneous temporary files
per calling thread (the MSDN documentation says 26, but that's a lie).
Once there are 42 files in your TEMP directory created by a single
Emacs session, all the future calls to 'mktemp' from the same session
will fail, unless you manually delete those files.

This part of the bug is harder to solve, because some code needs to be
implemented that will defer deletion until the process exits.  So it
is not solved yet.  But at least you now have a work-around: manually
delete temporary em* files from your TEMP directory.