bug#13065: Bug in x-file-dialog with GetOpenFileName

[Eli Zaretskii]

> From: Jason Rumney <jasonr@gnu.org>
> Cc: 13065@debbugs.gnu.org,  duyanning@gmail.com
> Date: Wed, 16 Jan 2013 20:23:19 +0800
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Failing that, I'm beginning to think that the problem is elsewhere in
> > Emacs.  But what could that be?  Is it possible that one of our
> > message pumps somehow interferes with the dialog interaction, e.g., by
> > intercepting some of the messages that were supposed to wind up in our
> > hook procedure?
> 
> That seems like the most likely candidate. Do you have Spy++, or an
> equivalent that can show the Windows messages going to and from your
> sample program and the Open/Save dialog in Emacs?
> 
> > Could it be due to the fact that our main message
> > pump uses Unicode APIs, while the file selector uses ANSI APIs?
> 
> Since that's quite a recent change, it should be easy to test by trying
> an older version of Emacs on Windows 7.

Yes, I also had the idea to check older versions.  It was a good idea,
because it allowed me to find the reason for the problem.  It wasn't
an interference between message pumps.  It was memory.  Here are the
details.

In preparation for Emacs 24.1, we changed the amount of memory we
reserve at startup: where Emacs 23 asked the OS to reserve 256MB, we
now try to reserve as much as possible, starting with 2GB and
gradually decreasing the amount until the OS grants the request.  On
XP, we typically end up with 1.1 to 1.5GB, but on Windows 7 we get
more: about 1.8GB.

Now, when we launch the file selector dialog, Windows starts several
threads in our process to support that (I see 4 threads on XP and
between 6 and 7 on Windows 7).  My guess is that with so much of the
address space reserved, these threads fail to reserve enough memory
for themselves, probably for their stack (which is set to 8MB in
Emacs), and misbehave as result of that.  We already had a similar
problem with the threads we ourselves create to support subprocesses,
see
http://lists.gnu.org/archive/html/emacs-devel/2012-03/msg00119.html.
In that case, we were able to fix the problem without decreasing the
amount of memory we ask at startup.  But here, these are threads that
are started outside our control, so a similar solution is not
possible, AFAIU.

So, unless someone has a better idea, I will soon commit to the
emacs-24 branch the patch shown below.  It fixes the problem on
Windows 7 and does not affect XP in any visible way.  The initial
amount of memory to reserve was found by trial and error.

(After this patch, the original recipe still doesn't work: you need to
type "abc" _after_ clicking "Desktop".  But the same happens with
Emacs 23, so I guess this is just a minor change in behavior of
Windows 7.)

=== modified file 'src/w32heap.c'
--- src/w32heap.c       2013-01-01 09:11:05 +0000
+++ src/w32heap.c       2013-01-16 18:34:38 +0000
@@ -98,7 +98,11 @@ allocate_heap (void)
 #ifdef _WIN64
   size_t size = 0x4000000000i64; /* start by asking for 32GB */
 #else
-  size_t size = 0x80000000; /* start by asking for 2GB */
+  /* We used to start with 2GB here, but on Windows 7 that would leave
+     too little room in the address space for threads started by
+     Windows on our behalf, e.g. when we pop up the file selection
+     dialog.  */
+  size_t size = 0x68000000; /* start by asking for 1.7GB */
 #endif
   void *ptr = NULL;