Emacs 23.0.60 build problem on CANNOT_DUMP platform GNUstep

[Chris]

Aloha, folks,

It seems emacs-pretest-bug list is no longer active, so I decided to 
send this here.

This was originally sent to the emacs-app-dev-@lists.sourceforge.net.  
The folks there helped come up with a workaround/possible solution, 
and YAMAMOTO Mitsuharu on that list suggested I send a report to 
emacs-pretest-bug.

I am running Debian Sarge (all updates current as of 22 Jan 08), with 
a custom 2.6.19 kernel.

* gcc (GCC) 3.3.5 (Debian 1:3.3.5-13)
* GNU make 3.8.0
* GNU gdb 6.3-debian
* xfree86 4.3.0.dfsg.1-14sarge7
* GNUstep: gnustep-back-0.12.1 gnustep-gui-0.12.1 gnustep-base-0.14.2 
gnustep-make-2.0.4

While trying to build Emacs.app rc3, a seg fault occured , here is the 
'tail' of the output:

===================================
mkdir -p /home/cjh/GNUstep/Build/emacs-
23.0.0_NS-9.0rc3/src/../nextstep/build/Emacs.app/
cp -f emacs /home/cjh/GNUstep/Build/emacs-
23.0.0_NS-9.0rc3/src/../nextstep/build/Emacs.app/Emacs
make[1]: Leaving directory `/home/cjh/GNUstep/Build/emacs-
23.0.0_NS-9.0rc3/src'
(export PARALLEL; PARALLEL=0; cd leim; make all  \
  CC='gcc' CFLAGS='-g -O3' CPPFLAGS='-D_BSD_SOURCE  ' \
  LDFLAGS='-Wl,-znocombreloc' MAKE='make')
make[1]: Entering directory `/home/cjh/GNUstep/Build/emacs-
23.0.0_NS-9.0rc3/leim'
EMACSLOADPATH=/home/cjh/GNUstep/Build/emacs-23.0.0_NS-9.0rc3/leim/../lispLC_ALL=C
.../src/emacs -batch --no-init-file --no-site-file --multibyte -f
batch-byte-compile quail/CCDOSPY.el
make[1]: *** [quail/CCDOSPY.elc] Segmentation fault
make[1]: Leaving directory `/home/cjh/GNUstep/Build/emacs-
23.0.0_NS-9.0rc3/leim'
make: *** [leim] Error 2
*** Compilation failed. ***
Please examine the above output to determine what went wrong,
edit the configure options in this script (\'compile\') to fix it, and
rerun.
===================================

I tried to run ../src/emacs standalone, it seg faulted.

I ran the '../src/emacs' under gdb, and obtained the following output:

================================================
(gdb) run  -batch --no-init-file --no-site-file --multibyte -f
batch-byte-compile quail/CCDOSPY.el
Starting program:
/home/cjh/GNUstep/Build/emacs-23.0.0_NS-9.0rc3/src/emacs -batch
--no-init-file --no-site-file --multibyte -f batch-byte-compile
quail/CCDOSPY.el
[Thread debugging using libthread_db enabled]
[New Thread -1222926208 (LWP 23067)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread -1222926208 (LWP 23067)]
Ffuncall (nargs=3, args=0xbf253060) at eval.c:2949
2949    {

(gdb) bt 15
#0  Ffuncall (nargs=3, args=0xbf253060) at eval.c:2949
#1  0x08197a1e in call2 (fn=0, arg1=138835841, arg2=140451101) at
eval.c:2837
#2  0x08195e8b in Fsignal (error_symbol=138835841, data=140451101) at
eval.c:1663
#3  0x081960a6 in xsignal (error_symbol=140451101, data=140451101) at
eval.c:1736
#4  0x0819610e in xsignal1 (error_symbol=140451101, arg=140451101) at
eval.c:1753
#5  0x08197ca7 in Ffuncall (nargs=3, args=0xbf253190) at eval.c:3096
#6  0x08197a1e in call2 (fn=0, arg1=138835841, arg2=140451093) at
eval.c:2837
#7  0x08195e8b in Fsignal (error_symbol=138835841, data=140451093) at
eval.c:1663
#8  0x081960a6 in xsignal (error_symbol=140451101, data=140451101) at
eval.c:1736
#9  0x0819610e in xsignal1 (error_symbol=140451101, arg=140451101) at
eval.c:1753
#10 0x08197ca7 in Ffuncall (nargs=3, args=0xbf2532c0) at eval.c:3096
#11 0x08197a1e in call2 (fn=0, arg1=138835841, arg2=140451085) at
eval.c:2837
#12 0x08195e8b in Fsignal (error_symbol=138835841, data=140451085) at
eval.c:1663
#13 0x081960a6 in xsignal (error_symbol=140451101, data=140451101) at
eval.c:1736
#14 0x0819610e in xsignal1 (error_symbol=140451101, arg=140451101) at
eval.c:1753
(More stack frames follow...)

(gdb) bt -5
#137711 0x0819b79e in Flength (sequence=138836081) at fns.c:183
#137712 0x0819cc05 in concat (nargs=1, args=0xbfa4f340,
target_type=Lisp_Cons, last_special=0)
     at fns.c:567
#137713 0x0819c2db in Fcopy_sequence (arg=138441645) at fns.c:496
#137714 0x081d2a51 in set_initial_environment () at callproc.c:1668
#137715 0x0811cacf in main (argc=8, argv=0xbfa4f704) at emacs.c:1576
(gdb)
================================================

Further research in gdb showed that 'Flength' invoked 
'wrong_argument_type', which started the uncontrolled looping leading 
to more than 137K stack frames.

I have lost the output from that session, but will recreate it, if you 
would like to see it.

I noticed that an 'if' statement in 'set_initial_environment()' might 
have been 'ifdef'ed out.

Here is the source of that function:
=====================================================
void
set_initial_environment ()
{
register char **envp;
#ifndef CANNOT_DUMP
if (initialized)
#endif
{
for (envp = environ; *envp; envp++)
Vprocess_environment = Fcons (build_string (*envp),
Vprocess_environment);
/* Ideally, the `copy' shouldn't be necessary, but it seems it's
frequent
to use `delete' and friends on process-environment. */
Vinitial_environment = Fcopy_sequence (Vprocess_environment);
}
}
=====================================================

A little grepping and Emacs TAGS goodness led us to this in
'config.h:969':

=====================================================
/* If we're using NeXTstep, define some consequences. */
#ifdef HAVE_NS
#define HAVE_WINDOW_SYSTEM
#define MULTI_KBOARD
#define HAVE_MOUSE
#ifdef GNUSTEP
#define CANNOT_DUMP
#endif
/* PENDING: These are used tor the Carbon port only. */
/* #undef MAC_OS */
/* #undef MAC_OSX */
#endif
======================================================

So, GNUstep is apparently a 'CANNOT_DUMP' platform.  I have no idea 
what this might mean, but
YAMAMOTO Mitsuharu at the emacs-app-dev- list seemed to find it 
significant -- based on that he suggested that I modify 
'set_initial_environment()' to initialize the 'Vprocess_environment' 
variable to 'Qnil'.

This seems to work -- the build completes normally.

The inserted line is isolated below:
======================================================
#ifndef CANNOT_DUMP
if (initialized)
#endif
{

Vprocess_environment = Qnil;

for (envp = environ; *envp; envp++)
Vprocess_environment = Fcons (build_string (*envp),
======================================================