bug#11662: trunk bootstrap segfaults with nonzero MALLOC_PERTURB_

[Paul Eggert]

This follows up on Jim Meyering's message at
<http://lists.gnu.org/archive/html/emacs-devel/2012-06/msg00159.html>.

On 06/09/2012 09:47 AM, Jim Meyering wrote:
> Just to let you know that I'm once again seeing temacs segfault
> when MALLOC_PERTURB_ is nonzero.  I was able to bootstrap by with
> MALLOC_PERTURB_=0, but with it set to a nonzero value, temacs gets the
> (sporadically usual, over the years, now) segfault.
> 
> This is on Fedora 17/x86_64 using gcc version 4.8.0 20120604.
> 
> When I use F17's current default gcc (version 4.7.0 20120507),
> (still with nonzero MALLOC_PERTURB_) I get a slightly different failure:
> 
>     make[3]: *** [dired-aux.elc] Segmentation fault (core dumped)

I ran with "export MALLOC_PERTURB_=219; export MALLOC_CHECK=3" on Fedora 15
and GCC 4.7.0, with an abort() in memory_full(), and got a similar failure,
with backtrace noted below.  This points in the neighborhood of the
recently-introduced vector allocation code, so I'll CC: this to Dmitry.
Note that there is no bug when compiling without optimization; the bug
occurs with the default optimization (-O2) but is not present if -O2
is not given.

Here's the patch I used to get the abort:

=== modified file 'src/alloc.c'
--- src/alloc.c 2012-06-08 09:58:43 +0000
+++ src/alloc.c 2012-06-09 22:13:15 +0000
@@ -493,6 +493,7 @@
 
   /* This used to call error, but if we've run out of memory, we could
      get infinite recursion trying to build the string.  */
+  abort ();
   xsignal (Qnil, Vmemory_signal_data);
 }
 
@@ -3800,6 +3801,7 @@
 
   /* This used to call error, but if we've run out of memory, we could
      get infinite recursion trying to build the string.  */
+  abort ();
   xsignal (Qnil, Vmemory_signal_data);
 }
 
and here's the backtrace:

(gdb) where
#0  0x0000003b308354f7 in kill () from /lib64/libc.so.6
#1  0x00000000004fcdfa in fatal_error_signal (sig=<optimized out>)
    at emacs.c:368
#2  fatal_error_signal (sig=<optimized out>) at emacs.c:338
#3  <signal handler called>
#4  0x0000003b308354f7 in kill () from /lib64/libc.so.6
#5  0x00000000004fc475 in abort () at emacs.c:396
#6  0x0000000000557f3a in memory_full (nbytes=nbytes@entry=201808)
    at alloc.c:3804
#7  0x0000000000558afb in lisp_malloc (nbytes=nbytes@entry=201808, 
    type=type@entry=MEM_TYPE_VECTORLIKE) at alloc.c:938
#8  0x000000000055978d in allocate_vectorlike (len=len@entry=25224)
    at alloc.c:3269
#9  0x0000000000559b73 in allocate_vectorlike (len=25224) at alloc.c:3257
#10 allocate_vector (len=len@entry=25224) at alloc.c:3298
#11 0x000000000057f5e3 in larger_vector (vec=16894405, 
    incr_min=incr_min@entry=8310, nitems_max=nitems_max@entry=-1)
    at fns.c:3445
#12 0x000000000057fffb in maybe_resize_hash_table (h=0xe1b358) at fns.c:3752
#13 hash_put (h=h@entry=0xe1b358, key=key@entry=18256518, 
    value=value@entry=11975170, hash=<optimized out>) at fns.c:3838
#14 0x0000000000580213 in Fputhash (key=key@entry=18256518, value=11975170, 
    table=<optimized out>) at fns.c:4509
#15 0x000000000058ab45 in print_preprocess (obj=18256518) at print.c:1183
#16 0x000000000058ac55 in print_preprocess (obj=18256550) at print.c:1199
#17 0x000000000058ab8e in print_preprocess (obj=obj@entry=13484741)
    at print.c:1211
#18 0x000000000058ed9f in print (obj=obj@entry=13484741, 
    printcharfun=printcharfun@entry=11975122, escapeflag=escapeflag@entry=1)
    at print.c:1085
#19 0x000000000058ff78 in Fprin1 (object=13484741, printcharfun=11975122)
    at print.c:570
#20 0x0000000000573129 in Ffuncall (nargs=<optimized out>, 
    args=args@entry=0x7fff24bc5db0) at eval.c:2824
#21 0x00000000005a9c70 in exec_byte_code (bytestr=<optimized out>, 
    vector=14932781, maxdepth=<optimized out>, args_template=<optimized out>, 
    nargs=nargs@entry=6, args=<optimized out>, args@entry=0xda40c8)
    at bytecode.c:784
#22 0x0000000000572b69 in funcall_lambda (fun=14932781, nargs=nargs@entry=6, 
    arg_vector=0xda40c8, arg_vector@entry=0x7fff24bc5f88) at eval.c:2987
#23 0x0000000000572f8b in Ffuncall (nargs=7, args=args@entry=0x7fff24bc5f80)
    at eval.c:2882
#24 0x00000000005a9c70 in exec_byte_code (bytestr=<optimized out>, 
    vector=15012229, maxdepth=<optimized out>, args_template=<optimized out>, 
    nargs=nargs@entry=5, args=<optimized out>, args@entry=0xda4a20)
    at bytecode.c:784
#25 0x0000000000572b69 in funcall_lambda (fun=15012229, nargs=nargs@entry=5, 
    arg_vector=0xda4a20, arg_vector@entry=0x7fff24bc6130) at eval.c:2987
#26 0x0000000000572f8b in Ffuncall (nargs=6, args=args@entry=0x7fff24bc6128)
    at eval.c:2882
#27 0x00000000005a9c70 in exec_byte_code (bytestr=<optimized out>, 
    vector=14513013, maxdepth=<optimized out>, args_template=<optimized out>, 
    nargs=nargs@entry=6, args=<optimized out>, args@entry=0xebb808)
    at bytecode.c:784
#28 0x0000000000572b69 in funcall_lambda (fun=14513013, nargs=nargs@entry=6, 
    arg_vector=0xebb808, arg_vector@entry=0x7fff24bc62f0) at eval.c:2987
#29 0x0000000000572f8b in Ffuncall (nargs=7, args=args@entry=0x7fff24bc62e8)
    at eval.c:2882
#30 0x00000000005a9c70 in exec_byte_code (bytestr=<optimized out>, 
    vector=12360997, maxdepth=<optimized out>, args_template=<optimized out>, 
    nargs=nargs@entry=5, args=<optimized out>, args@entry=0xebb770)
    at bytecode.c:784
#31 0x0000000000572b69 in funcall_lambda (fun=12360997, nargs=nargs@entry=5, 
    arg_vector=0xebb770, arg_vector@entry=0x7fff24bc64b0) at eval.c:2987
#32 0x0000000000572f8b in Ffuncall (nargs=6, args=args@entry=0x7fff24bc64a8)
    at eval.c:2882
#33 0x00000000005a9c70 in exec_byte_code (bytestr=<optimized out>, 
    vector=17842589, maxdepth=<optimized out>, args_template=<optimized out>, 
    nargs=nargs@entry=5, args=<optimized out>, args@entry=0xebb700)
    at bytecode.c:784
#34 0x0000000000572b69 in funcall_lambda (fun=17842589, nargs=nargs@entry=5, 
    arg_vector=0xebb700, arg_vector@entry=0x7fff24bc66a0) at eval.c:2987
#35 0x0000000000572f8b in Ffuncall (nargs=6, args=args@entry=0x7fff24bc6698)
    at eval.c:2882
#36 0x00000000005a9c70 in exec_byte_code (bytestr=<optimized out>, 
    vector=15727621, maxdepth=<optimized out>, args_template=<optimized out>, 
    nargs=nargs@entry=1, args=<optimized out>, args@entry=0xebb520)
    at bytecode.c:784
#37 0x0000000000572b69 in funcall_lambda (fun=15727621, nargs=nargs@entry=1, 
    arg_vector=0xebb520, arg_vector@entry=0x7fff24bc6860) at eval.c:2987
#38 0x0000000000572f8b in Ffuncall (nargs=2, args=args@entry=0x7fff24bc6858)
    at eval.c:2882
#39 0x00000000005a9c70 in exec_byte_code (bytestr=<optimized out>, 
    vector=17469325, maxdepth=<optimized out>, args_template=<optimized out>, 
    nargs=nargs@entry=1, args=<optimized out>, args@entry=0xda44e8)
    at bytecode.c:784
#40 0x0000000000572b69 in funcall_lambda (fun=17469325, nargs=nargs@entry=1, 
    arg_vector=0xda44e8, arg_vector@entry=0x7fff24bc69b8) at eval.c:2987
#41 0x0000000000572f8b in Ffuncall (nargs=2, args=args@entry=0x7fff24bc69b0)
    at eval.c:2882
#42 0x00000000005a9c70 in exec_byte_code (bytestr=<optimized out>, 
    vector=17469157, maxdepth=<optimized out>, args_template=<optimized out>, 
    nargs=nargs@entry=1, args=<optimized out>, args@entry=0xda4498)
    at bytecode.c:784
#43 0x0000000000572b69 in funcall_lambda (fun=17469157, nargs=nargs@entry=1, 
    arg_vector=0xda4498, arg_vector@entry=0x7fff24bc6b18) at eval.c:2987
#44 0x0000000000572f8b in Ffuncall (nargs=2, args=args@entry=0x7fff24bc6b10)
    at eval.c:2882
#45 0x00000000005a9c70 in exec_byte_code (bytestr=<optimized out>, 
    vector=14511245, maxdepth=<optimized out>, args_template=<optimized out>, 
    nargs=nargs@entry=0, args=<optimized out>, args@entry=0xda3730)
    at bytecode.c:784
#46 0x0000000000572b69 in funcall_lambda (fun=14511245, nargs=nargs@entry=0, 
    arg_vector=0xda3730, arg_vector@entry=0x7fff24bc6c70) at eval.c:2987
#47 0x0000000000572f8b in Ffuncall (nargs=1, args=args@entry=0x7fff24bc6c68)
    at eval.c:2882
#48 0x00000000005a9c70 in exec_byte_code (bytestr=<optimized out>, 
    vector=12420373, maxdepth=<optimized out>, args_template=<optimized out>, 
    nargs=nargs@entry=0, args=<optimized out>, args@entry=0xda3960)
    at bytecode.c:784
#49 0x0000000000572b69 in funcall_lambda (fun=12420373, nargs=nargs@entry=0, 
    arg_vector=0xda3960, arg_vector@entry=0x7fff24bc6db8) at eval.c:2987
#50 0x0000000000572f8b in Ffuncall (nargs=1, args=0x7fff24bc6db0)
    at eval.c:2882
#51 0x000000000057277a in eval_sub (form=form@entry=13010646) at eval.c:2140
#52 0x000000000057531f in internal_lisp_condition_case (var=15414098, 
    bodyform=13010646, handlers=13010358) at eval.c:1288
#53 0x00000000005aa5a3 in exec_byte_code (bytestr=<optimized out>, 
    vector=<optimized out>, maxdepth=<optimized out>, 
    args_template=<optimized out>, nargs=nargs@entry=1, args=<optimized out>, 
    args@entry=0xda3608) at bytecode.c:980
#54 0x0000000000572b69 in funcall_lambda (fun=15013133, nargs=nargs@entry=1, 
    arg_vector=0xda3608, arg_vector@entry=0x7fff24bc7240) at eval.c:2987
#55 0x0000000000572f8b in Ffuncall (nargs=2, args=args@entry=0x7fff24bc7238)
    at eval.c:2882
#56 0x00000000005a9c70 in exec_byte_code (bytestr=<optimized out>, 
    vector=15528981, maxdepth=<optimized out>, args_template=<optimized out>, 
    nargs=nargs@entry=1, args=<optimized out>, args@entry=0xda2eb0)
    at bytecode.c:784
#57 0x0000000000572b69 in funcall_lambda (fun=15528981, nargs=nargs@entry=1, 
    arg_vector=0xda2eb0, arg_vector@entry=0x7fff24bc7400) at eval.c:2987
#58 0x0000000000572f8b in Ffuncall (nargs=2, args=args@entry=0x7fff24bc73f8)
    at eval.c:2882
#59 0x00000000005a9c70 in exec_byte_code (bytestr=<optimized out>, 
    vector=15533581, maxdepth=<optimized out>, args_template=<optimized out>, 
    nargs=nargs@entry=0, args=<optimized out>, args@entry=0xeb6148)
    at bytecode.c:784
#60 0x0000000000572b69 in funcall_lambda (fun=15533581, nargs=nargs@entry=0, 
    arg_vector=0xeb6148, arg_vector@entry=0x7fff24bc7558) at eval.c:2987
#61 0x0000000000572f8b in Ffuncall (nargs=1, args=0x7fff24bc7550)
    at eval.c:2882
#62 0x000000000057277a in eval_sub (form=form@entry=12894454) at eval.c:2140
#63 0x000000000057531f in internal_lisp_condition_case (var=12521906, 
    bodyform=12894454, handlers=12895078) at eval.c:1288
#64 0x00000000005aa5a3 in exec_byte_code (bytestr=<optimized out>, 
    vector=<optimized out>, maxdepth=<optimized out>, 
    args_template=<optimized out>, nargs=nargs@entry=1, args=<optimized out>, 
    args@entry=0xeb60e0) at bytecode.c:980
#65 0x0000000000572b69 in funcall_lambda (fun=14589877, nargs=nargs@entry=1, 
    arg_vector=0xeb60e0, arg_vector@entry=0x7fff24bc7990) at eval.c:2987
#66 0x0000000000572f8b in Ffuncall (nargs=2, args=args@entry=0x7fff24bc7988)
    at eval.c:2882
#67 0x00000000005a9c70 in exec_byte_code (bytestr=<optimized out>, 
    vector=14589445, maxdepth=<optimized out>, args_template=<optimized out>, 
    nargs=nargs@entry=0, args=<optimized out>, args@entry=0xeb5fd8)
    at bytecode.c:784
#68 0x0000000000572b69 in funcall_lambda (fun=14589445, nargs=nargs@entry=0, 
    arg_vector=0xeb5fd8, arg_vector@entry=0x7fff24bc7b80) at eval.c:2987
#69 0x0000000000572f8b in Ffuncall (nargs=1, args=args@entry=0x7fff24bc7b78)
    at eval.c:2882
#70 0x00000000005a9c70 in exec_byte_code (bytestr=<optimized out>, 
    vector=9219269, maxdepth=<optimized out>, args_template=<optimized out>, 
    nargs=nargs@entry=1, args=<optimized out>, args@entry=0xa9d337)
    at bytecode.c:784
#71 0x0000000000572b69 in funcall_lambda (fun=9219269, nargs=nargs@entry=1, 
    arg_vector=0xa9d337, arg_vector@entry=0x7fff24bc7d00) at eval.c:2987
#72 0x0000000000572f8b in Ffuncall (nargs=2, args=args@entry=0x7fff24bc7cf8)
    at eval.c:2882
#73 0x00000000005a9c70 in exec_byte_code (bytestr=<optimized out>, 
    vector=9194589, maxdepth=<optimized out>, args_template=<optimized out>, 
    nargs=nargs@entry=0, args=<optimized out>, args@entry=0xa9f877)
    at bytecode.c:784
#74 0x0000000000572b69 in funcall_lambda (fun=9194589, nargs=nargs@entry=0, 
    arg_vector=0xa9f877, arg_vector@entry=0x7fff24bc7ed8) at eval.c:2987
#75 0x0000000000572f8b in Ffuncall (nargs=1, args=args@entry=0x7fff24bc7ed0)
    at eval.c:2882
#76 0x00000000005a9c70 in exec_byte_code (bytestr=<optimized out>, 
    vector=9189173, maxdepth=<optimized out>, args_template=<optimized out>, 
    nargs=nargs@entry=0, args=<optimized out>, args@entry=0xaa05f0)
    at bytecode.c:784
#77 0x0000000000572b69 in funcall_lambda (fun=9189173, fun@entry=9189085, 
    nargs=nargs@entry=0, arg_vector=0xaa05f0, arg_vector@entry=0x7fff24bc7fb0)
    at eval.c:2987
#78 0x000000000057205c in apply_lambda (fun=9189085, args=<optimized out>)
    at eval.c:2930
#79 0x0000000000572413 in eval_sub (form=form@entry=12488790) at eval.c:2232
#80 0x0000000000574c65 in Feval (form=12488790, lexical=<optimized out>)
    at eval.c:2022
#81 0x0000000000571553 in internal_condition_case (
    bfun=bfun@entry=0x4ffd00 <top_level_2>, handlers=12027074, 
    hfun=hfun@entry=0x5015a0 <cmd_error>) at eval.c:1334
#82 0x0000000000500236 in top_level_1 (ignore=ignore@entry=11975122)
    at keyboard.c:1172
#83 0x000000000057144b in internal_catch (
    tag=<error reading variable: Cannot access memory at address 0x5ffa>, 
    func=func@entry=0x5001d0 <top_level_1>, arg=11975122) at eval.c:1091
#84 0x000000000050105f in command_loop () at keyboard.c:1127
#85 recursive_edit_1 () at keyboard.c:754
#86 0x000000000050139a in Frecursive_edit () at keyboard.c:818
#87 0x00000000004171f8 in main (argc=9, argv=<optimized out>) at emacs.c:1699

Lisp Backtrace:
"prin1" (0x24bc5db8)
"byte-compile-output-docform" (0x24bc5f88)
"byte-compile-file-form-defmumble" (0x24bc6130)
0xeffae8 PVEC_COMPILED
0xeffb58 PVEC_COMPILED
0xeffbc8 PVEC_COMPILED
"byte-compile-file-form-defalias" (0x24bc6860)
"byte-compile-file-form" (0x24bc69b8)
"byte-compile-toplevel-file-form" (0x24bc6b18)
0xecaf60 PVEC_COMPILED
0xdd6d38 PVEC_COMPILED
"funcall" (0x24bc6db0)
"byte-compile-from-buffer" (0x24bc7240)
"byte-compile-file" (0x24bc7400)
0xdeb840 PVEC_COMPILED
"funcall" (0x24bc7550)
"batch-byte-compile-file" (0x24bc7990)
"batch-byte-compile" (0x24bc7b80)
"command-line-1" (0x24bc7d00)
"command-line" (0x24bc7ed8)
"normal-top-level" (0x24bc7fb0)