[gnugo-devel] 3.0 portability problem

[Gunnar Farneback]

I have tried building and running GNU Go 3.0.0 on a Cray Y-MP/EL. This
is a computer that some 10 years ago or so was a speed monster (but
not technically classified as a supercomputer, I believe), at least
for vector computations. Today it spends it days in retirement at
Lysator, the academic computer society in Link?ping, Sweden.

GNU Go 3.0.0 builds cleanly, in some 40 minutes. The problems come
with the regressions. After thinking for 252 hours (yes, I'm serious,
it took a week and a half) it came up with these results:

nobel% time make all_batches
cmd-2429 make: No suffix list.
        ./regress.sh . reading.tst
        ./regress.sh . owl.tst
        ./regress.sh . owl_rot.tst
        ./regress.sh . ld_owl.tst
        ./regress.sh . optics.tst
        ./regress.sh . life.tst --life
        ./regress.sh . filllib.tst
20 unexpected FAIL: Correct 'A13', got 'S7'
25 unexpected FAIL: Correct 'J3|J5', got 'H1'
        ./regress.sh . atari_atari.tst
        ./regress.sh . connection.tst
        ./regress.sh . blunder.tst
2 unexpected FAIL: Correct 'C9|H9', got 'E9'
4 unexpected FAIL: Correct 'B5|C5|C4|D4|E4|E3|F3', got 'B7'
8 unexpected FAIL: Correct '!(A3|A5)', got 'A3'
11 unexpected FAIL: Correct 'D4|E4|E5|F4|C6', got 'B3'
        ./regress.sh . strategy.tst
1 unexpected FAIL: Correct 'Q13', got 'P17'
5 unexpected FAIL: Correct 'R17|P17', got 'E14'
9 unexpected FAIL: Correct 'P15', got 'O15'
12 unexpected FAIL: Correct 'N4', got 'L4'
14 unexpected FAIL: Correct 'E9', got 'B3'
16 unexpected FAIL: Correct 'H12', got 'T4'
21 unexpected FAIL: Correct 'S13', got 'T12'
22 unexpected FAIL: Correct 'K3', got 'H13'
27 unexpected FAIL: Correct 'C3', got 'Q7'
29 unexpected FAIL: Correct 'R4', got 'O15'
32 unexpected FAIL: Correct 'A4', got 'G8'
33 unexpected FAIL: Correct 'O7', got 'R4'
39 unexpected FAIL: Correct 'P16', got 'D5'
40 unexpected FAIL: Correct 'E5', got 'D5'
42 unexpected FAIL: Correct 'C17', got 'N6'
45 unexpected FAIL: Correct 'G17|G18|C1', got 'Q7'
49 unexpected FAIL: Correct 'C17|D16|B17', got 'J14'
        ./regress.sh . strategy2.tst
51 unexpected FAIL: Correct 'P3', got 'D7'
53 unexpected FAIL: Correct 'S15', got 'A6'
54 unexpected FAIL: Correct 'P6', got 'L4'
55 unexpected FAIL: Correct 'C12', got 'T12'
59 unexpected FAIL: Correct 'B1', got 'C1'
60 unexpected FAIL: Correct 'H7', got 'J2'
61 unexpected FAIL: Correct 'M2', got 'M3'
65 unexpected FAIL: Correct 'M17', got 'Q7'
73 unexpected PASS!
76 unexpected FAIL: Correct 'Q11', got 'F7'
79 unexpected FAIL: Correct 'N3', got 'L4'
81 unexpected FAIL: Correct 'E13|D13', got 'H11'
85 unexpected FAIL: Correct 'O2', got 'D7'
90 unexpected FAIL: Correct 'B14|D17', got 'J14'
91 unexpected FAIL: Correct 'B14', got 'J14'
92 unexpected FAIL: Correct 'G13', got 'J14'
94 unexpected FAIL: Correct 'A2', got 'N18'
95 unexpected FAIL: Correct '!P13', got 'P13'
99 unexpected PASS!
        ./regress.sh . endgame.tst
101 unexpected FAIL: Correct 'F9', got 'C5'
102 unexpected FAIL: Correct 'E9', got 'E8'
103 unexpected FAIL: Correct 'G9', got 'C5'
105 unexpected FAIL: Correct 'J2|J3|A7', got 'C5'
108 unexpected FAIL: Correct 'G1', got 'H1'
109 unexpected FAIL: Correct 'A7', got 'C5'
203 unexpected FAIL: Correct 'A7', got 'T17'
204 unexpected FAIL: Correct 'H19|A12', got 'L4'
205 unexpected FAIL: Correct 'F18|E18', got 'T17'
206 unexpected FAIL: Correct 'A12', got 'L4'
209 unexpected FAIL: Correct 'A6|O1', got 'T17'
210 unexpected FAIL: Correct 'A5', got 'N3'
211 unexpected FAIL: Correct 'A5', got 'O2'
213 unexpected FAIL: Correct 'Q8|C13', got 'T17'
214 unexpected FAIL: Correct 'R8|C12|C13', got 'L4'
215 unexpected FAIL: Correct 'C13', got 'T17'
216 unexpected FAIL: Correct 'D13', got 'L4'
217 unexpected FAIL: Correct 'C12', got 'T17'
218 unexpected FAIL: Correct 'H1', got 'L4'
401 unexpected PASS!
        ./regress.sh . heikki.tst
10 unexpected FAIL: Correct 'M18', got 'E14'
        ./regress.sh . neurogo.tst
1 unexpected FAIL: Correct 'Q3', got 'M13'
2 unexpected FAIL: Correct 'R3', got 'M13'
8 unexpected FAIL: Correct 'O2', got 'A6'
18 unexpected FAIL: Correct 'K19|H18|K16', got 'N19'
        ./regress.sh . arb.tst
200 unexpected FAIL: Correct 'F14', got 'K12'
220 unexpected PASS!
231 unexpected FAIL: Correct 'F5', got 'K2'
        ./regress.sh . rosebud.tst
        ./regress.sh . golife.tst
2 unexpected FAIL: Correct 'H5', got 'A6'
3 unexpected FAIL: Correct 'E3|E2', got 'C5'
4 unexpected FAIL: Correct 'H5', got 'C5'
5 unexpected FAIL: Correct 'H5', got 'C5'
8 unexpected FAIL: Correct 'C2|G1', got 'G5'
        ./regress.sh . arion.tst
1 unexpected FAIL: Correct 'Q6', got 'F7'
2 unexpected FAIL: Correct 'Q18|S16', got 'E14'
4 unexpected FAIL: Correct 'F9', got 'L4'
        ./regress.sh . viking.tst
3 unexpected FAIL: Correct 'M15|B7', got 'M13'
5 unexpected FAIL: Correct 'S5', got 'A6'
6 unexpected FAIL: Correct 'B17|B18', got 'A6'
        ./regress.sh . ego.tst
1 unexpected FAIL: Correct 'K4|K3', got 'P2'
3 unexpected FAIL: Correct 'H18', got 'Q7'
4 unexpected PASS!
9 unexpected FAIL: Correct 'S19', got 'L4'
12 unexpected FAIL: Correct 'R10|C12', got 'D10'
        ./regress.sh . dniwog.tst
1 unexpected FAIL: Correct 'D15', got 'D7'
4 unexpected FAIL: Correct 'D7', got 'Q7'
        ./regress.sh . nicklas1.tst
203 unexpected FAIL: Correct 'J5', got 'C6'
204 unexpected FAIL: Correct 'F8', got 'J5'
301 unexpected FAIL: Correct 'H3|H6', got 'J8'
501 unexpected FAIL: Correct 'G7', got 'G8'
502 unexpected FAIL: Correct 'G5', got 'H6'
801 unexpected FAIL: Correct 'P16', got 'D5'
1002 unexpected FAIL: Correct 'H8', got 'E4'
1206 unexpected FAIL: Correct 'C15', got 'E14'
1208 unexpected FAIL: Correct 'E19', got 'J14'
1213 unexpected FAIL: Correct 'N4', got 'K13'
1216 unexpected FAIL: Correct 'H3', got 'D2'
1405 unexpected FAIL: Correct 'B7', got 'B6'
1502 unexpected FAIL: Correct 'D14', got 'N1'
1506 unexpected FAIL: Correct 'S4', got 'T1'
1507 unexpected FAIL: Correct '!T1', got 'T1'
1508 unexpected FAIL: Correct '!F15', got 'F15'
1510 unexpected FAIL: Correct 'C12', got 'D5'
1603 unexpected FAIL: Correct 'B4', got 'C5'
1604 unexpected PASS!
1901 unexpected FAIL: Correct 'B7', got 'E4'
2001 unexpected FAIL: Correct 'G5', got 'J4'
2301 unexpected FAIL: Correct 'G6|F7', got 'E6'
2302 unexpected FAIL: Correct 'F4', got 'D6'
        ./regress.sh . nicklas2.tst
102 unexpected PASS!
1001 unexpected FAIL: Correct 'B7', got 'E4'
1402 unexpected PASS!
1701 unexpected PASS!
1802 unexpected FAIL: Correct 'E2', got 'E6'
        ./regress.sh . nicklas3.tst
        ./regress.sh . nicklas4.tst
        ./regress.sh . nicklas5.tst
804 unexpected FAIL: Correct 'D6', got 'Q7'
1202 unexpected PASS!
1203 unexpected PASS!
        ./regress.sh . strategy3.tst
103 unexpected FAIL: Correct 'B16', got 'A17'
104 unexpected FAIL: Correct 'P4', got 'L4'
106 unexpected FAIL: Correct 'F10', got 'R1'
107 unexpected FAIL: Correct 'F1', got 'R1'
108 unexpected FAIL: Correct 'C15', got 'A10'
110 unexpected FAIL: Correct 'A8', got 'R6'
112 unexpected FAIL: Correct 'M12', got 'E14'
113 unexpected FAIL: Correct 'P1', got 'O15'
115 unexpected FAIL: Correct 'G2', got 'C9'
118 unexpected FAIL: Correct 'A8', got 'J6'
119 unexpected FAIL: Correct 'D9|J3', got 'J6'
121 unexpected FAIL: Correct 'C7', got 'C9'
122 unexpected FAIL: Correct '(R19|T18)', got 'O15'
123 unexpected FAIL: Correct 'P4', got 'Q7'
128 unexpected FAIL: Correct 'O8', got 'M3'
134 unexpected PASS!
135 unexpected PASS!
137 unexpected FAIL: Correct 'F5', got 'E14'
145 unexpected FAIL: Correct 'S14|T14', got 'E10'
149 unexpected PASS!
151 unexpected FAIL: Correct 'T5', got 'D5'
152 unexpected FAIL: Correct 'S8', got 'D5'
154 unexpected PASS!
156 unexpected FAIL: Correct 'B14|B16', got 'E10'
162 unexpected PASS!
164 unexpected PASS!
        ./regress.sh . manyfaces.tst
1 unexpected FAIL: Correct 'S13', got 'S6'
2 unexpected FAIL: Correct 'S16', got 'Q6'
        ./regress.sh . niki.tst
1 unexpected FAIL: Correct 'E17', got 'D7'
4 unexpected FAIL: Correct 'Q8', got 'J14'
13 unexpected PASS!
909133.63u 3917.19s 253:51:21.12 99.9%

Not at all satisfactory. A little debugging showed that the random
numbers weren't working correctly. This in turn was caused by the fact
that the machine has 64 bit ints (in contrast to some other 64 bit
platforms where an int still is 32 bits while a long is 64 bits) and
the code to detect this in 3.0.0 has a bug. This can be solved either
by including config.h in random.c or by applying the following patch,
which I think is a better solution. This patch of course applies to
both 3.0 and 3.1. I've added it in CVS for the latter line.

After patching, the regressions pass cleanly on the CRAY Y-MP/EL. (In
only about four days, by running the three regression batches in
parallel on three of the four available processors.)

/Gunnar

Index: random.c
===================================================================
RCS file: /cvsroot/gnugo/gnugo/utils/random.c,v
retrieving revision 1.3
diff -u -r1.3 random.c
--- random.c    2001/09/06 21:34:21     1.3
+++ random.c    2001/10/09 08:53:00
@@ -58,6 +58,12 @@
 /* Set when properly seeded. */
 static int rand_initialized = 0;
 
+/* We use this to detect whether unsigned ints are bigger than 32
+ * bits. If they are we need to clear higher order bits, otherwise we
+ * can optimize by not doing the masking.
+ */
+#define BIG_UINT (UINT_MAX > 0xffffffffU)
+
 
 /* Iterate the TGFSR once to get a new state which can be used to
  * produce another 25 random numbers.
@@ -91,7 +97,7 @@
   }
   y = x[k] ^ ((x[k] << s) & b);
   y ^= ((y << t) & c);
-#if (CHAR_BIT * SIZEOF_INT > 32)
+#if BIG_UINT
   y &= 0xffffffffU;
 #endif
   return y;
@@ -114,7 +120,7 @@
 {
   int i;
   for (i=0; i<N; i++) {
-#if (CHAR_BIT * SIZEOF_INT > 32)
+#if BIG_UINT
     seed &= 0xffffffffU;
 #endif
     x[i] = seed;