[gnugo-devel] Regression file suggestion & patch

[Trevor Morris]

Here's a patch that does the following:
 - Cleans up a few typos & mistakes in the current regression suite.
 - Replaces several occurences of genmove_black w/ gg_genmove black.
 - New GoBoard::Stone Perl Module for generate PNG files.
 - Updates regress.pl to generate graphics, not text in HTML view.
 - Implements the following suggestion in several places.

http://www.public32.com/games/go/trevor_1_9.4.tar.gz
[This file includes the patch file, one new SGF file, and one
new Perl module for generating stone images.]

Suggestion:
Add the following 3 comment lines to existing failure cases:
#CATEGORY={the program area (of a defined set) that needs fixing.
  Current options: 
 ("JOSEKI_DATABASE, "",
  "JOSEKI_PATTERN", "", 
  "FUSEKI_CONCEPT", "",

"DYNAMIC_CONNECTION", "Dynamic Connection Reading",
  "TACTICAL_READING",
"",
  "OWL_TUNING", "",
  "PATTERN_TUNING", "",
  "CONNECTION_TUNING", "",

 "MOVE_VALUATION", "",
  "ATARI_ATARI", "",
  "SEMEAI_MODULE", "",

"KO_READING", "")
#DESCRIPTION=A short one-line description of the
problem.
#SEVERITY={0-10} 0:ignore; 1:low priority; 10:Very important.

For example:
#CATEGORY=OWL_TUNING
#DESCRIPTION=W group needs help & 2 black stones
attacking.
#SEVERITY=8

#CATEGORY=PATTERN_TUNING
#DESCRIPTION=No terribly serious consequences, S11
not optimal.
#SEVERITY=3

The idea is that regress.pl will sort the failures appropriately
into areas, sorted by severity.


Here's the most interesting part of the patch text, but don't 
try to apply it - use the gz file listed above, instead:

Index: NICKLAS
===================================================================
RCS file: /cvsroot/gnugo/gnugo/regression/NICKLAS,v
retrieving revision 1.3
diff -u -r1.3 NICKLAS
--- NICKLAS     2001/09/06 21:34:20     1.3
+++ NICKLAS     2001/10/05 17:51:37
@@ -1,3 +1,8 @@
+****GnuGo 3.1.8 still misses these 4 problems.
+****10/5/01 - moved comments into nicklas2.tst.
+****This file now should be considerred obsolete.
+
+
 nicklas1.tst: all these tests should pass
 nicklas2.tst: 9x9 games where gnugo should do better.
 nicklas3.tst: 9x9 games with harder problems
@@ -8,7 +13,7 @@
                             NICKLAS2.TST
 
 #102: LOW PRIORITY; DIFFICULTY HARD; DEFER
-gnugo --quiet -l niclas1.sgf -L29 -t
+gnugo --quiet -l nicklas1.sgf -L29 -t
 
     This is not as clear cut as it may seem. 2.7.228 values the endgame 
     move at A7 to 6.66 points which seems about correct. The fact that 
Index: dniwog.tst
===================================================================
RCS file: /cvsroot/gnugo/gnugo/regression/dniwog.tst,v
retrieving revision 1.3
diff -u -r1.3 dniwog.tst
--- dniwog.tst  2001/09/06 21:34:20     1.3
+++ dniwog.tst  2001/10/05 17:51:37
@@ -16,14 +16,24 @@
 4 gg_genmove white
 #? [D7]
 
+#CATEGORY=OWL_TUNING
+#DESCRIPTION=W group needs help & 2 black stones attacking.
+#SEVERITY=8
 loadsgf games/dniwog.sgf 78
 5 gg_genmove white
 #? [H4|G4]*
 
+
+#CATEGORY=PATTERN_TUNING
+#DESCRIPTION=A more clear-cut case for this tesuji might help.
+#SEVERITY=7
 loadsgf games/dniwog.sgf 88
 6 gg_genmove white
 #? [G1]*
 
+#CATEGORY=PATTERN_TUNING
+#DESCRIPTION=Safing right side group is better than terrible gote move!
+#SEVERITY=10
 loadsgf games/dniwog.sgf 96
 7 gg_genmove white
 #? [B8]*
Index: ego.tst
===================================================================
RCS file: /cvsroot/gnugo/gnugo/regression/ego.tst,v
retrieving revision 1.7
diff -u -r1.7 ego.tst
--- ego.tst     2001/09/15 00:24:22     1.7
+++ ego.tst     2001/10/05 17:51:37
@@ -13,11 +13,18 @@
 2 gg_genmove white
 #? [!E5]
 
+
+#CATEGORY=PATTERN_TUNING
+#DESCRIPTION=K17 misses H18 threat.
+#SEVERITY=8
 # K18 is unreasonable. H18 is the only move locally.
 loadsgf games/ego.sgf 52
 3 gg_genmove white
 #? [H18]*
 
+#CATEGORY=PATTERN_TUNING
+#DESCRIPTION=Possible replacement pattern appropriate?
+#SEVERITY=8
 # The descent at G19 is bad shape because it leaves much aji around
 # E18 and gives black an endgame sente move at H19.
 loadsgf games/ego.sgf 54
@@ -38,6 +45,10 @@
 7 gg_genmove white
 #? [!Q1]
 
+
+#CATEGORY=BORDER_PROTECTION
+#DESCRIPTION=Tough moyo protection.
+#SEVERITY=5
 loadsgf games/ego.sgf 112
 8 gg_genmove white
 #? [F12]*
@@ -50,6 +61,10 @@
 10 gg_genmove white
 #? [S18]
 
+
+#CATEGORY=KO_READING
+#DESCRIPTION=Tough - Black pushthrough is troublesome.
+#SEVERITY=3
 loadsgf games/ego.sgf 198
 11 gg_genmove white
 #? [T18]*
Index: manyfaces.tst
===================================================================
RCS file: /cvsroot/gnugo/gnugo/regression/manyfaces.tst,v
retrieving revision 1.3
diff -u -r1.3 manyfaces.tst
--- manyfaces.tst       2001/09/06 21:34:20     1.3
+++ manyfaces.tst       2001/10/05 17:51:38
@@ -5,43 +5,61 @@
 reset_trymove_counter
 
 loadsgf games/mfgg1.sgf 43
-1 genmove_black
+1 gg_genmove black
 #? [S13]
 
 loadsgf games/mfgg1.sgf 49
-2 genmove_black
+2 gg_genmove black
 #? [S16]
 
+#CATEGORY=PATTERN_TUNING
+#DESCRIPTION=No terribly serious consequences, S11 not optimal.
+#SEVERITY=3
 loadsgf games/mfgg1.sgf 65
-3 genmove_black
+3 gg_genmove black
 #? [S12]*
 
 loadsgf games/mfgg1.sgf 97
-4 genmove_black
+4 gg_genmove black
 #? [!T6]
 
+#CATEGORY=FUSEKI_STRATEGY
+#DESCRIPTION=N6 is not terrible - right concept.
+#SEVERITY=2
 loadsgf games/mfgg2.sgf 19
-5 genmove_black
+5 gg_genmove black
 #? [M5|M15|R3|R17]*
 
+
+#CATEGORY=FUSEKI_STRATEGY
+#DESCRIPTION=E6 really misses the point.
+#SEVERITY=9
 loadsgf games/mfgg2.sgf 23
-6 genmove_black
+6 gg_genmove black
 #? [R3]*
 
+#CATEGORY=OWL_TUNING
+#DESCRIPTION=Q5 dragon is under severe pressure.
+#SEVERITY=8
 loadsgf games/mfgg2.sgf 45
-7 genmove_black
+7 gg_genmove black
 #? [R2|S7]*
 
+
+#CATEGORY=OWL_TUNING
+#DESCRIPTION=GnuGo doesn't understand this type of corner position!
+#SEVERITY=10
+#GG often messes up in this type of corner position.
 loadsgf games/mfgg2.sgf 117
-8 genmove_black
+8 gg_genmove black
 #? [T2]*
 
 loadsgf games/mfgg3.sgf 42
-9 genmove_black
+9 gg_genmove black
 #? [!P12]
 
 loadsgf games/mfgg3.sgf 44
-10 genmove_black
+10 gg_genmove black
 #? [N3]*
 
 
Index: nicklas1.tst
===================================================================
RCS file: /cvsroot/gnugo/gnugo/regression/nicklas1.tst,v
retrieving revision 1.9
diff -u -r1.9 nicklas1.tst
--- nicklas1.tst        2001/10/01 03:49:08     1.9
+++ nicklas1.tst        2001/10/05 17:51:41
@@ -21,6 +21,9 @@
 202 gg_genmove black
 #? [!H8]
 
+#CATEGORY=OWL_TUNING
+#DESCRIPTION=Combination threat to live / connect.
+#SEVERITY=4
 loadsgf games/nicklas/nicklas2.sgf 23
 203 gg_genmove black
 #? [J5]*
@@ -74,10 +77,13 @@
 810 gg_genmove black
 #? [!C19]
 
-# Moved K15 to K14. /gf
+# Moved K15 to K14. /gf ; Added back K14 - both work.
+#SEVERITY=7
+#CATEGORY=CONNECTION_TUNING
+#DESCRIPTION=Is GnuGo misreading the cut?
 loadsgf games/nicklas/nicklas8.sgf 240
 811 gg_genmove black
-#? [K14]*
+#? [K14|K15]*
 
 loadsgf games/nicklas/nicklas9.sgf 28
 901 gg_genmove black
@@ -119,6 +125,9 @@
 1106 gg_genmove black
 #? [!F6]
 
+#CATEGORY=WASTED_MOVE
+#DESCRIPTION=GnuGo 3.1.8 gets N2 on next move;  R3 benign ko threat loss.
+#SEVERITY=2
 loadsgf games/nicklas/nicklas11.sgf 242
 1107 gg_genmove black
 #? [N2]*
@@ -151,6 +160,9 @@
 1206 gg_genmove black
 #? [C15]
 
+#CATEGORY=PATTERN_TUNING
+#DESCRIPTION=Needless 2 point loss for B.
+#SEVERITY=7
 loadsgf games/nicklas/nicklas12.sgf 113
 1207 gg_genmove black
 #? [C17]*
@@ -268,6 +280,9 @@
 1605 gg_genmove black
 #? [J1]
 
+#CATEGORY=OWL_TUNING
+#DESCRIPTION=No reason to give W a chance for ko here.
+#SEVERITY=6
 loadsgf games/nicklas/nicklas18.sgf 15
 1801 gg_genmove black
 #? [D2]*
Index: nicklas2.tst
===================================================================
RCS file: /cvsroot/gnugo/gnugo/regression/nicklas2.tst,v
retrieving revision 1.8
diff -u -r1.8 nicklas2.tst
--- nicklas2.tst        2001/09/24 14:05:44     1.8
+++ nicklas2.tst        2001/10/05 17:51:42
@@ -11,18 +11,50 @@
 reset_reading_node_counter
 reset_trymove_counter
 
+#CATEGORY=ATARI_ATARI
+#DESCRIPTION=GnuGo misses followup to D4 push-through.
+#SEVERITY=1
+#    This is not as clear cut as it may seem. 2.7.228 values the endgame 
+#    move at A7 to 6.66 points which seems about correct. The fact that 
+#    white D4 threatens both to cut off E5-F6 and to capture D3 through 
+#    a chain of threats (which is what happens in the game) is not that
+#    easy to understand.  A more general atari_atari module could 
+#    perhaps see it, but we are still far from that.  I think this test
+#    should be in nicklas3.tst (harder problems).
 loadsgf games/nicklas/nicklas1.sgf 29
 102 gg_genmove black
 #? [D4]*
 
+#CATEGORY=PATTERN_TUNING
+#DESCRIPTION=I think G9 is not too terrible here. Very difficult to
see.-trevor
+#SEVERITY=1
+#    This test illustrates a general weakness of GNU Go: It's poor 
+#    ability to defend its borders.  White can enter blacks domains or
+#    capture the E8 group. In the latter case, whites first move is
+#    a sacrifice and the capture happens through a chain of threats.
+#Added E6, still not thinking G9 is so bad. -trevor
 loadsgf games/nicklas/nicklas6.sgf 23
 601 gg_genmove black
-#? [D8|C8|D6|D4]*
+#? [D8|C8|D6|D4|E6]*
 
+#CATEGORY=SEMEAI_MODULE
+#DESCRIPTION=Very tough position.  Even if F1 for B, W E7 is tough.
+#SEVERITY=1
+#    This is a very good test case for a semeai module. Black F1 threatens
+#    to live and creates all the necessary liberties to win the semai.
+#I disagree; this it tough; a more clear-cut test case would be helpful
+#to better isolate the inherent problem. - trevor
 loadsgf games/nicklas/nicklas7.sgf 29
 701 gg_genmove black
 #? [F1]*
 
+
+#CATEGORY=OWL_TUNING
+#DESCRIPTION=Tough for white to see attacking B, and linking w/ attack stone.
+#SEVERITY=3
+#    The only way to live is to attack a neighbour dragon, e.g. the one
+#    at F8. But does the owl code really do this by default?
+#
 loadsgf games/nicklas/nicklas9.sgf 28
 902 dragon_status E9
 #? [critical]*
@@ -45,35 +77,68 @@
 1301 gg_genmove black
 #? [B8|B9]
 
+#CATEGORY=TACTICAL_READING
+#DESCRIPTION=Fighting Ko here is not unreasonable for B.
+#SEVERITY=0
+#I would delete this regression test. -trevor
 loadsgf games/nicklas/nicklas14.sgf 31
 1401 gg_genmove black
 #? [B3]*
 
+#CATEGORY=OWL_TUNING
+#DESCRIPTION=Black must protect his group.  H9 is thinkable also.
+#SEVERITY=9
 loadsgf games/nicklas/nicklas14.sgf 43
 1402 gg_genmove black
 #? [J8|J6]*
 
+#CATEGORY=OWL_TUNING
+#DESCRIPTION=PASS here is unthinkable, though RESIGN might be appropriate!
+#SEVERITY=5
 # One eye wins against no eye
+#Note: Owl & Dragon status is both critical for the W group.  Why
+#  would B pass when there's a critical group on the board? Owl & 
+#  Dragon status for all of the B stones is Dead.  Can a Critical
+#  group kill another group???
 loadsgf games/nicklas/nicklas14.sgf 61
 1407 gg_genmove black
 #? [A6]*
 
+
+#CATEGORY=PATTERN_TUNING
+#DESCRIPTION=Protect those borders!
+#SEVERITY=8
+#Added A8, for the faint of heart.
 loadsgf games/nicklas/nicklas17.sgf 47
 1701 gg_genmove black
-#? [C9]*
+#? [C9|A8]*
 
 loadsgf games/nicklas/nicklas18.sgf 17
 1802 gg_genmove black
 #? [E2]
 
+
+#CATEGORY=WASTED_MOVE
+#DESCRIPTION=Um, you're still dead after B1, but sente!
+#SEVERITY=1
+#SEE_ALSO=trevor.tst?310
 loadsgf games/nicklas/nicklas18.sgf 45
 1803 gg_genmove black
 #? [A8]*
 
+
+#CATEGORY=ENDGAME_TUNING
+#DESCRIPTION=OWL & Dragon status for W is all dead.  Why does B play?
+#SEVERITY=5
 loadsgf games/nicklas/nicklas21.sgf 72
 2102 gg_genmove black
 #? [PASS]*
 
+
+#CATEGORY=ENDGAME_TUNING
+#DESCRIPTION=OWL & Dragon status for W is all dead.  Why does B play?
+#SEVERITY=5
+#See also nicklas2.tst?2102
 loadsgf games/nicklas/nicklas21.sgf 74
 2103 gg_genmove black
 #? [PASS]*
@@ -90,6 +155,9 @@
 2202 gg_genmove black
 #? [B1]
 
+#CATEGORY=ATARI_ATARI
+#DESCRIPTION=Black must defend here - yikes!
+#SEVERITY=10
 loadsgf games/nicklas/nicklas24.sgf 41
 2401 gg_genmove black
 #? [G3]*
Index: niki.tst
===================================================================
RCS file: /cvsroot/gnugo/gnugo/regression/niki.tst,v
retrieving revision 1.5
diff -u -r1.5 niki.tst
--- niki.tst    2001/10/02 20:04:18     1.5
+++ niki.tst    2001/10/05 17:51:42
@@ -9,10 +9,16 @@
 1 gg_genmove white
 #? [E17]
 
+#CATEGORY=MIDDLE_STRATEGY
+#DESCRIPTION=Tough middle-game position
+#SEVERITY=3
 loadsgf games/niki.sgf 40
 2 gg_genmove white
 #? [E7]*
 
+#CATEGORY=PATTERN_TUNING
+#DESCRIPTION=Must connect 5 important stones, cutting B group.
+#SEVERITY=10
 loadsgf games/niki.sgf 68
 3 gg_genmove white
 #? [E12]*
@@ -21,14 +27,20 @@
 4 gg_genmove white
 #? [Q8]
 
+#S10 seems OK, much better than game move of S14. Added it -trevor
+#Hm, should GG3.1.9 get credit for this? I think so! -trevor
 loadsgf games/niki.sgf 94
 5 gg_genmove white
-#? [S9]*
+#? [S9|S10]*
 
 loadsgf games/niki.sgf 106
 6 gg_genmove white
 #? [!S18]
 
+
+#CATEGORY=PATTERN_TUNING
+#DESCRIPTION=T4 is terrible; other moves than J12 maybe OK.
+#SEVERITY=6
 loadsgf games/niki.sgf 110
 7 gg_genmove white
 #? [J12]*
@@ -36,6 +48,9 @@
 loadsgf games/niki.sgf 124
 8 gg_genmove white
 #? [G16]*
+
+#gg_genmove bug requires reloading game.
+loadsgf games/niki.sgf 124
 9 gg_genmove black
 #? [F15]*
 
@@ -43,13 +58,25 @@
 10 gg_genmove white
 #? [S5]*
 
+
+#CATEGORY=TACTICAL_READING
+#DESCRIPTION=Very tough position
+#SEVERITY=3
+#Note: G16 is still very very big.  See niki.tst#8 & nikit.tst#9
 # FIXME: Same move listed twice.
 loadsgf games/niki.sgf 158
 11 gg_genmove white
 #? [M6|M6|P7]*
 
+
+#CATEGORY=OWL_TUNING
+#DESCRIPTION=B15 does live.  Problem is w/ followup - see niki.tst#14
+#SEVERITY=0
+#
 # Originally H12 was listed as correct here, but that vertex is
 # occupied. F14 and G15 seem sufficient for life.
+#
+#Recommend adding B15 as acceptable to live.
 loadsgf games/niki.sgf 212
 12 gg_genmove white
 #? [F14|G15]*
@@ -57,6 +84,14 @@
 loadsgf games/niki.sgf 226
 13 gg_genmove white
 #? [F8]
+
+
+#CATEGORY=OWL_TUNING
+#DESCRIPTION=F16 lives because of B short of liberties
+#SEVERITY=8
+loadsgf games/niki.sgf 214
+14 gg_genmove white
+#? [F16]*
 
 # Report number of nodes visited by the tactical reading
 10000 get_reading_node_counter
Index: strategy.tst
===================================================================
RCS file: /cvsroot/gnugo/gnugo/regression/strategy.tst,v
retrieving revision 1.8
diff -u -r1.8 strategy.tst
--- strategy.tst        2001/10/01 03:49:08     1.8
+++ strategy.tst        2001/10/05 17:51:46
@@ -5,7 +5,7 @@
 reset_trymove_counter
 
 loadsgf games/strategy1.sgf
-1 genmove_white
+1 gg_genmove white
 #? [Q13]
 
 loadsgf games/strategy2.sgf 5
Index: tactics.tst
===================================================================
RCS file: /cvsroot/gnugo/gnugo/regression/tactics.tst,v
retrieving revision 1.2
diff -u -r1.2 tactics.tst
--- tactics.tst 2001/10/01 03:49:08     1.2
+++ tactics.tst 2001/10/05 17:51:46
@@ -5,7 +5,7 @@
 reset_trymove_counter
 
 loadsgf games/tactics01.sgf
-1 genmove_black
+1 gg_genmove black
 #? [C15|D15|B13|B11|B14]*
 
 ############ End of Tests #################
Index: trevor.tst
===================================================================
RCS file: /cvsroot/gnugo/gnugo/regression/trevor.tst,v
retrieving revision 1.5
diff -u -r1.5 trevor.tst
--- trevor.tst  2001/10/03 20:57:57     1.5
+++ trevor.tst  2001/10/05 17:51:48
@@ -305,6 +305,9 @@
 301 gg_genmove black
 #? [!PASS]*
 
-
-
-
+#CATEGORY=OWL_TUNING
+#DESCRIPTION=Black still dead after D1!
+#SEVERITY=10
+loadsgf games/trevor/trevor_31.sgf
+310 gg_genmove black
+#? [A8]*