[Bug-gnulib] stdbool_.h bug breaks coreutils CVS sort on Alpha GCC 2.95.4

[Paul Eggert]

Jim Meyering <address@hidden> writes:

> I've just tested on the sourceforge compile farm alpha system
> using 2.95.4 and noticed that sort was segfaulting.
> Turns out it's due to what I suppose must be a bug in
> gcc's handling of this new code:
> 
>       bool swap = (0 < compare (&lines[-1], &lines[-2]));
>       temp[-1] = lines[-1 - swap];
>       temp[-2] = lines[-2 + swap];
> 
> if I change the `bool' to int, it works:...
> 
> gdb shows what's happening:...
>
>   (gdb) p -1 - false
>   $2 = 4294967295


Ouch.  It's most likely a gnulib bug, not a GCC bug.  GCC 2.95.4
doesn't support _Bool, so the substitute stdbool.h supplied by gnulib
does this:

  typedef enum { false = 0, true = 1 } _Bool;

My guess is that on your platform GCC 2.95.4 assigns to _Bool a type
that is compatible with unsigned int.  Hence "-1 - false" is
equivalent to "-1 - (unsigned int) 0", which evaluates to (unsigned
int) -1, or 4294967295.

GCC 2.95.4's behavior conforms to C89, so this is a bug in gnulib's
stdbool_.h.  Can you please try the enclosed patch on that platform?
I'll CC: this message to bug-gnulib, since it's clearly a stdbool_.h
portability bug in general, even if I've happened to misdiagnose the
problem on your particular platform.

Here's a simple program that illustrates the bug in stdbool_.h.
It should exit with status 0, but with the gnulib stdbool_.h
it exits with status 1 (even on a 32-bit platform).

        #include <stdbool.h>
        bool False = 0;
        int main (void)
        {
          return 0 < -1 - False;
        }

Here's a patch to coreutils; the gnulib patch is a subset of this.

2003-07-28  Paul Eggert  <address@hidden>

        * lib/stdbool.hin (_Bool): Make it signed char, instead of
        an enum type, so that it's guaranteed to promote to int.
        * src/sort.c (sortlines_temp): Undo previous change.

diff -pru coreutils/lib/stdbool.hin coreutils-bool/lib/stdbool.hin
--- coreutils/lib/stdbool.hin   Mon Jun  2 01:15:16 2003
+++ coreutils-bool/lib/stdbool.hin      Mon Jul 28 00:02:22 2003
@@ -29,10 +29,14 @@
 # undef true
 #endif
 
-/* For the sake of symbolic names in gdb, define _Bool as an enum type.  */
+/* For the sake of symbolic names in gdb, define true and false as
+   enum constants.  However, do not define _Bool as the enum type,
+   since the enum type might be compatible with unsigned int, whereas
+   _Bool must promote to int.  */
 #ifndef __cplusplus
 # if address@hidden@
-typedef enum { false = 0, true = 1 } _Bool;
+enum { false = 0, true = 1 };
+typedef signed char _Bool;
 # endif
 #else
 typedef bool _Bool;
diff -pru coreutils/src/sort.c coreutils-bool/src/sort.c
--- coreutils/src/sort.c        Sun Jul 27 15:05:43 2003
+++ coreutils-bool/src/sort.c   Sun Jul 27 23:56:04 2003
@@ -1863,7 +1863,7 @@ sortlines_temp (struct line *lines, size
 {
   if (nlines == 2)
     {
-      int swap = (0 < compare (&lines[-1], &lines[-2]));
+      bool swap = (0 < compare (&lines[-1], &lines[-2]));
       temp[-1] = lines[-1 - swap];
       temp[-2] = lines[-2 + swap];
     }