bug#18222: 24.3.92; fork handlers in gmalloc.c can lead to deadlock

[Ken Brown]

On 8/9/2014 3:44 AM, YAMAMOTO Mitsuharu wrote:
> > > > > > On Fri, 08 Aug 2014 09:09:31 -0400, Ken Brown <kbrown@cornell.edu> said:
>
> > malloc_enable_thread() in gmalloc.c calls pthread_atfork to set up fork
> > handlers.  There are a couple of problems with this, but the immediate
> > reason for this bug report is a problem on Cygwin that was reported in
> > the thread starting at
>
> >     https://cygwin.com/ml/cygwin/2014-07/msg00387.html
>
> > and continuing at
>
> >     https://cygwin.com/ml/cygwin/2014-08/msg00001.html.
>
> > The issue is that the 'prepare' fork handler locks the pthread_mutexes
> > prior to forking, and the ensuing processing of the fork command by the
> > Cygwin DLL leads to a call to malloc in the same thread, resulting in
> > deadlock.  This is a long-standing problem, but it was masked until
> > recently by the fact that pthread_mutexes on Cygwin were ERRORCHECK
> > mutexes by default.  As of Cygwin 1.7.31, pthread_mutexes are now NORMAL
> > mutexes by default, so the problem has shown up.
>
> > A simple short-term workaround would be to explicitly set the mutexes to
> > be ERRORCHECK or RECURSIVE mutexes on Cygwin, thereby restoring the
> > previous behavior.  But this does not seem like the right long-term
> > solution, for the reasons explained here:
>
> >     https://cygwin.com/ml/cygwin/2014-08/msg00161.html
> >     https://cygwin.com/ml/cygwin/2014-08/msg00175.html
>
> > I know nothing about this other than what I learned from the two
> > messages above, so I would appreciate some guidance.
>
> Originally, gmalloc.c bundled with Emacs was not thread-safe, so I
> added some mutex code as a short-term solution:
>
>    http://lists.gnu.org/archive/html/emacs-devel/2007-06/msg01782.html
>
> Thread-safe malloc was required mainly for GLib (via GTK+, for
> example), and atfork handers were necessary because the threads
> internally used by GLib were not under our control.
>
> All the platforms I'm currently working at use their system malloc
> rather than Emacs's gmalloc.c, so I don't think I can be of much help
> about this issue.  If changing mutex attributes works well, then I
> think that would be good enough for a workaround for upcoming 24.4
> release.

OK.  For maximum safety, I should probably set the type of the mutexes 
to ERRORCHECK, as they were in Cygwin 1.7.30, even though I think 
RECURSIVE would work just as well.

> For a long term solution, it might be better to think about
> a transition to Cygwin's malloc

I'll try to do this, but it's not at all clear how, given the way emacs 
is built on Cygwin.  (See the comments at lines 309 and 1301 in gmalloc.c.)

> (you mentioned
> malloc_set_state/malloc_get_state in (*), but they are used only when
> DOUG_LEA_MALLOC is defined).

You're right, although there's something of a Catch 22 here.  Cygwin's 
malloc is in fact Doug Lea malloc, but the emacs configure test for 
DOUG_LEA_MALLOC simply tests for malloc_set_state and malloc_get_state 
(and a couple of hooks).

In any case, I think it would be much easier to switch to Cygwin's 
malloc if it had malloc_set_state and malloc_get_state.  I wonder how 
hard it would be to add those.

Ken