bug#22790: 24.5; Infinite loop involving malloc called from signal handler

[Andreas Gustafsson]

I'm using emacs and the VM package to read mail under NetBSD 6.1.5 on a
daily basis, often working with mailboxes many tens of megabytes in
size.  Once every few days of doing this, emacs becomes unresponsive,
consuming 100% CPU.  I rebuilt emacs with debug symbols, and the last
time this happened, I caught the following backtrace:

(gdb) where
#0  0x00007f7ff6c08445 in ?? () from /usr/lib/libpthread.so.1
#1  0x00007f7ff6c08848 in ?? () from /usr/lib/libpthread.so.1
#2  0x00000000005c5486 in _malloc_internal (size=65536) at gmalloc.c:929
#3  0x00000000005c54fc in malloc (size=65536) at gmalloc.c:953
#4  0x00007f7ff60ed28c in __smakebuf () from /usr/lib/libc.so.12
#5  0x00007f7ff60ed125 in __swsetup () from /usr/lib/libc.so.12
#6  0x00007f7ff60cde92 in __vfprintf_unlocked () from /usr/lib/libc.so.12
#7  0x00007f7ff60d1258 in vfprintf () from /usr/lib/libc.so.12
#8  0x00007f7ff60cc266 in printf () from /usr/lib/libc.so.12
#9  0x00000000004db715 in handle_interrupt (in_signal_handler=true) at 
keyboard.c:10364
#10 0x00000000004db63e in handle_interrupt_signal (sig=2) at keyboard.c:10288
#11 0x00000000004e8b63 in deliver_process_signal (sig=2, handler=0x4db5f1 
<handle_interrupt_signal>) at sysdep.c:1570
#12 0x00000000004db65a in deliver_interrupt_signal (sig=2) at keyboard.c:10295
#13 <signal handler called>
#14 0x00007f7ff6c083e2 in ?? () from /usr/lib/libpthread.so.1
#15 0x00007f7ff6c08445 in ?? () from /usr/lib/libpthread.so.1
#16 0x00007f7ff6c08848 in ?? () from /usr/lib/libpthread.so.1
#17 0x00000000005c626f in _free_internal (ptr=0x33e0000) at gmalloc.c:1268
#18 0x00000000005c62ca in free (ptr=0x33e0000) at gmalloc.c:1283
#19 0x0000000000535046 in xfree (block=0x33e0000) at alloc.c:735
#20 0x000000000055ba6d in unbind_to (count=2, value=11946034) at eval.c:3304
#21 0x000000000055715d in unwind_to_catch (catch=0xfbf600, value=45248038) at 
eval.c:1161
#22 0x000000000055806a in Fsignal (error_symbol=12016098, data=11946034) at 
eval.c:1557
#23 0x00000000004db9c7 in handle_interrupt (in_signal_handler=true) at 
keyboard.c:10421
#24 0x00000000004db63e in handle_interrupt_signal (sig=2) at keyboard.c:10288
#25 0x00000000004e8b63 in deliver_process_signal (sig=2, handler=0x4db5f1 
<handle_interrupt_signal>) at sysdep.c:1570
#26 0x00000000004db65a in deliver_interrupt_signal (sig=2) at keyboard.c:10295
#27 <signal handler called>
#28 0x00007f7ff60ec172 in memmove () from /usr/lib/libc.so.12
#29 0x00000000005c7c7a in r_alloc_sbrk (size=8527872) at ralloc.c:821
#30 0x00000000005c42b7 in align (size=8527872) at gmalloc.c:423
#31 0x00000000005c46c9 in morecore_nolock (size=8527872) at gmalloc.c:624
#32 0x00000000005c4fc9 in _malloc_internal_nolock (size=8394752) at 
gmalloc.c:863
#33 0x00000000005c6571 in _realloc_internal_nolock (ptr=0x2be0000, 
size=8394752) at gmalloc.c:1424
#34 0x00000000005c66fe in _realloc_internal (ptr=0x2be0000, size=8394752) at 
gmalloc.c:1480
#35 0x00000000005c6773 in realloc (ptr=0x2be0000, size=8394752) at 
gmalloc.c:1495
#36 0x0000000000534fec in xrealloc (block=0x2be0000, size=8394752) at 
alloc.c:717
#37 0x0000000000473021 in coding_alloc_by_realloc (coding=0x7f7fffff9a70, 
bytes=6144) at coding.c:1061
#38 0x000000000047337c in alloc_destination (coding=0x7f7fffff9a70, 
nbytes=6144, dst=0x33df400 "") at coding.c:1102
#39 0x00000000004881f2 in encode_coding_raw_text (coding=0x7f7fffff9a70) at 
coding.c:5429
#40 0x0000000000490639 in encode_coding (coding=0x7f7fffff9a70) at coding.c:7802
#41 0x0000000000492467 in encode_coding_object (coding=0x7f7fffff9a70, 
src_object=40695813, from=8388609, from_byte=8407959, to=16777217, 
to_byte=16796567, dst_object=11946082)
    at coding.c:8371
#42 0x000000000050dcef in e_write (desc=6, string=11946034, start=8388609, 
end=52984520, coding=0x7f7fffff9a70) at fileio.c:5256
#43 0x000000000050d8e9 in a_write (desc=6, string=11946034, pos=1, 
nchars=52984519, annot=0x7f7fffff9da8, coding=0x7f7fffff9a70) at fileio.c:5172
#44 0x000000000050ce5f in write_region (start=4, end=211938080, 
filename=33678465, append=11946034, visit=11946082, lockname=33724897, 
mustbenew=11946034, desc=6) at fileio.c:4870
#45 0x000000000050c758 in Fwrite_region (start=11946034, end=11946034, 
filename=33678465, append=11946034, visit=11946082, lockname=33724897, 
mustbenew=11946034) at fileio.c:4679
#46 0x000000000055ab18 in Ffuncall (nargs=7, args=0x7f7fffffa070) at eval.c:2837
#47 0x0000000000599506 in exec_byte_code (bytestr=8987609, vector=8987645, 
maxdepth=76, args_template=0, nargs=0, args=0x7f7fffffa5b0) at bytecode.c:916
#48 0x000000000055b0c7 in funcall_lambda (fun=8987565, nargs=0, 
arg_vector=0x7f7fffffa5b0) at eval.c:2978
#49 0x000000000055abb1 in Ffuncall (nargs=1, args=0x7f7fffffa5a8) at eval.c:2860
#50 0x0000000000599506 in exec_byte_code (bytestr=8987417, vector=8987453, 
maxdepth=12, args_template=0, nargs=0, args=0x7f7fffffaab0) at bytecode.c:916
#51 0x000000000055b0c7 in funcall_lambda (fun=8987373, nargs=0, 
arg_vector=0x7f7fffffaab0) at eval.c:2978
#52 0x000000000055abb1 in Ffuncall (nargs=1, args=0x7f7fffffaaa8) at eval.c:2860
#53 0x0000000000599506 in exec_byte_code (bytestr=8986145, vector=8986181, 
maxdepth=40, args_template=0, nargs=0, args=0x7f7fffffafc0) at bytecode.c:916
#54 0x000000000055b0c7 in funcall_lambda (fun=8986093, nargs=0, 
arg_vector=0x7f7fffffafc0) at eval.c:2978
#55 0x000000000055abb1 in Ffuncall (nargs=1, args=0x7f7fffffafb8) at eval.c:2860
#56 0x0000000000599506 in exec_byte_code (bytestr=8984849, vector=8984885, 
maxdepth=20, args_template=1024, nargs=0, args=0x7f7fffffb4a8) at bytecode.c:916
#57 0x000000000055b0c7 in funcall_lambda (fun=8984797, nargs=0, 
arg_vector=0x7f7fffffb4a8) at eval.c:2978
#58 0x000000000055abb1 in Ffuncall (nargs=1, args=0x7f7fffffb4a0) at eval.c:2860
#59 0x0000000000599506 in exec_byte_code (bytestr=8989777, vector=8989813, 
maxdepth=8, args_template=1028, nargs=1, args=0x7f7fffffba20) at bytecode.c:916
#60 0x000000000055b0c7 in funcall_lambda (fun=8989733, nargs=1, 
arg_vector=0x7f7fffffba18) at eval.c:2978
#61 0x000000000055abb1 in Ffuncall (nargs=2, args=0x7f7fffffba10) at eval.c:2860
#62 0x0000000000599506 in exec_byte_code (bytestr=8723753, vector=8723789, 
maxdepth=140, args_template=6156, nargs=5, args=0x7f7fffffbfc0) at 
bytecode.c:916
#63 0x000000000055b0c7 in funcall_lambda (fun=8723709, nargs=5, 
arg_vector=0x7f7fffffbf98) at eval.c:2978
#64 0x000000000055abb1 in Ffuncall (nargs=6, args=0x7f7fffffbf90) at eval.c:2860
#65 0x0000000000599506 in exec_byte_code (bytestr=8989113, vector=8989149, 
maxdepth=64, args_template=2048, nargs=2, args=0x7f7fffffc4c8) at bytecode.c:916
#66 0x000000000055b0c7 in funcall_lambda (fun=8989061, nargs=2, 
arg_vector=0x7f7fffffc4b8) at eval.c:2978
#67 0x000000000055abb1 in Ffuncall (nargs=3, args=0x7f7fffffc4b0) at eval.c:2860
#68 0x0000000000599506 in exec_byte_code (bytestr=9007353, vector=9007389, 
maxdepth=64, args_template=1024, nargs=1, args=0x7f7fffffca10) at bytecode.c:916
#69 0x000000000055b0c7 in funcall_lambda (fun=9007301, nargs=1, 
arg_vector=0x7f7fffffca08) at eval.c:2978
#70 0x000000000055abb1 in Ffuncall (nargs=2, args=0x7f7fffffca00) at eval.c:2860
#71 0x0000000000599506 in exec_byte_code (bytestr=9008337, vector=9008373, 
maxdepth=16, args_template=1024, nargs=1, args=0x7f7fffffcf40) at bytecode.c:916
#72 0x000000000055b0c7 in funcall_lambda (fun=9008285, nargs=1, 
arg_vector=0x7f7fffffcf38) at eval.c:2978
#73 0x000000000055abb1 in Ffuncall (nargs=2, args=0x7f7fffffcf30) at eval.c:2860
#74 0x00000000005552cf in Fcall_interactively (function=16250050, 
record_flag=11946034, keys=11998845) at callint.c:836
#75 0x000000000055aa05 in Ffuncall (nargs=4, args=0x7f7fffffd238) at eval.c:2818
#76 0x0000000000599506 in exec_byte_code (bytestr=9460233, vector=9460269, 
maxdepth=52, args_template=4100, nargs=1, args=0x7f7fffffd750) at bytecode.c:916
#77 0x000000000055b0c7 in funcall_lambda (fun=9460189, nargs=1, 
arg_vector=0x7f7fffffd748) at eval.c:2978
#78 0x000000000055abb1 in Ffuncall (nargs=2, args=0x7f7fffffd740) at eval.c:2860
#79 0x000000000055a35f in call1 (fn=12009122, arg1=16250050) at eval.c:2610
#80 0x00000000004cb8a9 in command_loop_1 () at keyboard.c:1560
#81 0x0000000000557882 in internal_condition_case (bfun=0x4cb1f1 
<command_loop_1>, handlers=12016002, hfun=0x4cab3b <cmd_error>) at eval.c:1348
#82 0x00000000004caf5d in command_loop_2 (ignore=11946034) at keyboard.c:1178
#83 0x00000000005570b5 in internal_catch (tag=12008098, func=0x4caf37 
<command_loop_2>, arg=11946034) at eval.c:1112
#84 0x00000000004caf0f in command_loop () at keyboard.c:1157
#85 0x00000000004ca737 in recursive_edit_1 () at keyboard.c:778
#86 0x00000000004ca8a4 in Frecursive_edit () at keyboard.c:849
#87 0x00000000004c8aa4 in main (argc=4, argv=0x7f7fffffdb80) at emacs.c:1642
(gdb) 

For obvious reasons, the information below is not from the same emacs
process, but it is from the same binary.

In GNU Emacs 24.5.1 (x86_64--netbsd)
 of 2016-02-14 on guava.gson.org
Configured using:
 `configure --srcdir=/usr/pkgsrc/editors/emacs24/work/emacs-24.5
 --localstatedir=/var --without-dbus --without-gnutls --without-rsvg
 --without-x --without-xpm --without-jpeg --without-tiff --without-gif
 --without-png --prefix=/usr/pkg --build=x86_64--netbsd
 --host=x86_64--netbsd --infodir=/usr/pkg/info --mandir=/usr/pkg/man
 'CFLAGS=-g -I/usr/include -I/usr/pkg/include' 'CPPFLAGS=-DTERMINFO
 -I/usr/include -I/usr/pkg/include' 'LDFLAGS=-L/usr/lib -Wl,-R/usr/lib
 -L/usr/pkg/lib -Wl,-R/usr/pkg/lib''

Important settings:
  locale-coding-system: nil

Major mode: Debugger

Minor modes in effect:
  tooltip-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent messages:
Note: c-basic-offset adjusted to 2 for buffer keyboard.c.
Mark set [2 times]
Note: c-basic-offset adjusted to 2 for buffer sysdep.c.
Mark set
Mark saved where search started [2 times]
Mark set
scroll-up-command: End of buffer
Command: next 1
Undo!
Mark saved where search started

Load-path shadows:
/u/gson/lisp/tempo hides /usr/pkg/share/emacs/24.5/lisp/tempo
/usr/pkg/share/emacs/site-lisp/ispell/ispell hides 
/usr/pkg/share/emacs/24.5/lisp/textmodes/ispell

Features:
(shadow sort gnus-util mail-extr warnings emacsbug message format-spec
rfc822 mml mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums
mm-util help-fns mail-prsvr mail-utils cc-langs cc-mode cc-fonts
easymenu cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine misearch
multi-isearch gdb-mi cl-loaddefs cl-lib bindat json gud tool-bar
easy-mmode comint ansi-color ring xterm time-date guess-offset cc-vars
cc-defs regexp-opt tooltip electric uniquify ediff-hook vc-hooks
lisp-float-type tabulated-list newcomment lisp-mode prog-mode register
page menu-bar rfn-eshadow timer select mouse jit-lock font-lock syntax
facemenu font-core frame cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese hebrew greek romanian slovak
czech european ethiopic indian cyrillic chinese case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer nadvice loaddefs button
faces cus-face macroexp files text-properties overlay sha1 md5 base64
format env code-pages mule custom widget hashtable-print-readable
backquote make-network-process multi-tty emacs)

Memory information:
((conses 16 138264 9026)
 (symbols 48 20788 0)
 (miscs 40 86 618)
 (strings 32 21808 4914)
 (string-bytes 1 693311)
 (vectors 16 11410)
 (vector-slots 8 384549 3962)
 (floats 8 69 395)
 (intervals 56 2366 101)
 (buffers 960 19))