[gnugo-devel] Owl tuning

[Portela Fernand]

Hi,

I've spent a lot of time in the past weeks trying to better understand the
owl code and more specifically, the owl pattern databases. My attempts at
automated tunings were unfortunately not successful, but I gathered a lot
of informations and statistics. The result is a first manual tuning based
on these and a couple thoughts.

I'll start with the "thoughts". It is first to be noticed that there's a
large performance (in terms of successes/failures) difference between the
2 main databases. While the defense database in average is achieving about
66% success rate ('success' here actually means 'non-failure', that is a
move which doesn't give the opponent a WIN), the attacking one only gets
a bit more than 26% (!)

Some plausible explanations for this phenomenon :

1. the attacking database is less developped than the defense one

  Here, I don't have much to say. Maybe we could try to locate positions
  where the owl attacking code fails to suggest any move (I've seen some
  cases in the past), and fill the holes...

2. defense is overvalued because of the escape potential overestimation

  As said, the defense database performs at 66% globally, but the D13xx
  patterns alone are way over 80%. I think there are 2 factors here,
  related to points 3. et 4.

  When a defense pattern is proposing a jump, it is not very well checked
  that the stone is connected. I made an experiment with D1356 which led
  to a PASS in lazarus:6 (see below)

  Also, I think we should maybe rethink the attacking scheme globally.
  When I take some simple references, like in following :
  http://senseis.xmp.net/?ApproachingALifeAndDeathProblemTheRightWay
  it seems to me that we shouldn't generate any kind of placements /
  vital moves, as long as there still is an escape route available.
  Easier said than done, but I think we could try to work in this
  direction.

3. defense is overvalued because of the lack of understanding cuts in the
   goal dragon during the owl reading

  I made some attempts at this a couple weeks ago. I failed to code a
  proper re-build of the splitted goals. I intend to try again soon, with
  a direct use of the connection code, and see what happens with the
  performance...

4. attacks aren't very well planned

  As said above, too many moves are tried before the dragon has been
  properly confined. Any suggestions here to make a better job at this ?
  Maybe differentiate WINs by escape from others, and stop trying vital
  attack moves as soon as one of them failed because the goal escaped...

5. attacks are tried against obviously unkillable dragons

  I was about to propose a couple changes in the current code to prevent
  these wasteful attacks, but Arend's recent patch might very well change
  things a lot here, so better wait and see.


To the patch now. First a short note : I removed some patterns because of
their relative performance in the regressions. This is arguable and I'm not
forgetting that regression tests aren't fully representative. Still, if
someone wants to reactivate some of them, (s)he should be careful to
constrain them properly.

Breakage (compared to CVS before arend_3_13.7a) is rather light :
 9 PASSes and 2 FAILures.

The FAILs :

 handtalk:2
 strategy5:232

are both to be considered accidental if we take into account that the
simple increase of the owl node limit up to 1200 would solve them both.

The PASSes :

 golife:4               not sure, possibly lucky
 lazarus:6              clearly improved (see above)
 global:4               lucky and bad 
                        (being able to kill E4 or not shouldn't matter here)
 global:5               same as global:4
 13x13:37               very good
 13x13:52               good
 nngs4:130              unchecked
 century2002:110        excellent
 gunnar:4               O18 looks like a clear improvement to me (although,
                        there might be better moves on the board)


And finally about performance : a bit more than 3% owl nodes decrease, but
about 1.8% tactical nodes increase (probably caused by the change in D1356)

/nando


- owl tuning

Index: patterns/owl_attackpats.db
===================================================================
RCS file: /cvsroot/gnugo/gnugo/patterns/owl_attackpats.db,v
retrieving revision 1.69
diff -u -r1.69 owl_attackpats.db
--- patterns/owl_attackpats.db  8 Dec 2002 14:39:30 -0000       1.69
+++ patterns/owl_attackpats.db  14 Dec 2002 16:53:15 -0000
@@ -214,16 +214,19 @@
 :8,-,value(60)
 
 
-Pattern A107
-# gf Not on second line without known support. See trevora:290. (3.1.15)
-
-....?     block on third line
-Y.*.?
-....?
-....?
------
-
-:8,-,value(50)
+# nn Removed (3.3.13)
+#   Success rate in regressions : 2.88% (53/1786)
+# 
+# Pattern A107
+# # gf Not on second line without known support. See trevora:290. (3.1.15)
+# 
+# ....?           block on third line
+# Y.*.?
+# ....?
+# ....?
+# -----
+# 
+# :8,-,value(50)
 
 
 Pattern A108
@@ -483,22 +486,25 @@
 ;!x_somewhere(a) || attack(a)
 
 
-Pattern A207d
-# nn New pattern (3.3.11) penalty for being gote
-
-?O.Yxx         second line kosumi
-X.*.xx
-?...??
-------
-
-:8,-,value(60)
-
-?O.Yxx
-a.*.xx
-?...??
-------
-
-;!attack(a)
+# nn Removed (3.3.13)
+#   Success rate in regressions : 2.61% (15/560)
+# 
+# Pattern A207d
+# # nn New pattern (3.3.11) penalty for being gote
+# 
+# ?O.Yxx         second line kosumi
+# X.*.xx
+# ?...??
+# ------
+# 
+# :8,-,value(60)
+# 
+# ?O.Yxx
+# a.*.xx
+# ?...??
+# ------
+# 
+# ;!attack(a)
 
 
 # Pattern A208
@@ -1288,20 +1294,24 @@
 
 
 Pattern A406b
+# nn Modifed (3.3.13)
 
-xxx
-...         center block
-.*O
-.Y?
+?xxx?
+o...o         center block
+..*Oo
+o.Y??
 
 :8,-,value(80)
 
-xax
-...
-.*O
-.Y?
+?xax?
+o...o
+..*Od
+o.Ybc
 
 ;owl_escape_value(a)>0
+;&& (!x_somewhere(b,c) 
+;    || o_somewhere(b,d)
+;    || (o_somewhere(c) && !safe_xmove(d)))
 
 
 Pattern A406c
@@ -1722,7 +1732,6 @@
 ;(owl_escape_value(b) > 0)
 
 
-
 #########################################################
 #                                                       #
 #          Eye reducing moves on the second line        #
@@ -2161,24 +2170,27 @@
 ;lib(A) == 3
 
 
-Pattern A616
-# tm New Pattern (3.1.15)
-# tm Modified (3.1.17)
-
-?XOo|   special edge collapse
-?.Y.|
-?x.*|
-?x..|
-
-:8,s,value(45)
-
-?XCa|
-?.Y.|
-?b.*|
-?x..|
-
-; !attack(C)
-; && (!x_somewhere(b) || (x_somewhere(b) && o_somewhere(a)))
+# nn Removed (3.3.13)
+#   Success rate in regressions : 0.75 % (2/268)
+# 
+# Pattern A616
+# # tm New Pattern (3.1.15)
+# # tm Modified (3.1.17)
+# 
+# ?XOo|         special edge collapse
+# ?.Y.|
+# ?x.*|
+# ?x..|
+# 
+# :8,s,value(45)
+# 
+# ?XCa|
+# ?.Y.|
+# ?b.*|
+# ?x..|
+# 
+# ; !attack(C)
+# ; && (!x_somewhere(b) || (x_somewhere(b) && o_somewhere(a)))
 
 
 Pattern A617
@@ -3525,7 +3537,8 @@
 a*
 Yb
 
-; !xplay_defend_both(*,a,b) && vital_chain(a) && vital_chain(b)
+; vital_chain(a) && vital_chain(b)
+; && !xplay_defend_both(*,a,b)
 
 
 Pattern A1107b
@@ -3547,23 +3560,25 @@
 ;lib(c)>2 && vital_chain(b) && vital_chain(c)
 ;&& oplay_attack(*,a,a) && !xplay_defend_both(a,b,c)
 
-
-Pattern A1107c
-# tm New pattern. (3.1.17)
-# tm Modified (3.1.20) - reduced value (see nngs:320)
-
-.*       defend the surrounding chain
-O.
-YO
-
-:8,-,value(60)
-
-.*
-ba
-Yc
-
-;lib(c)>2 && vital_chain(b) && vital_chain(c)
-;&& oplay_attack(*,a,a) && !xplay_defend_both(a,b,c)
+# nn Removed (3.3.13)
+#   Success rate in regressions : 0.9 % (23/2562)
+#
+# Pattern A1107c
+# # tm New pattern. (3.1.17)
+# # tm Modified (3.1.20) - reduced value (see nngs:320)
+# 
+# .*       defend the surrounding chain
+# O.
+# YO
+# 
+# :8,-,value(60)
+# 
+# .*
+# ba
+# Yc
+# 
+# ;lib(c)>2 && vital_chain(b) && vital_chain(c)
+# ;&& oplay_attack(*,a,a) && !xplay_defend_both(a,b,c)
 
 
 Pattern A1108
@@ -4884,16 +4899,18 @@
 
 :8,-,value(46)
 
-
-Pattern A1402
-
-..Y..|       contest the corner territory
-*....|
-.....|
-.....|
------+
-
-:8,-,value(45)
+# nn Removed (3.3.13)
+#   Success rate in regressions : 0% (0/40)
+# 
+# Pattern A1402
+# 
+# ..Y..|       contest the corner territory
+# *....|
+# .....|
+# .....|
+# -----+
+# 
+# :8,-,value(45)
 
 
 #########################################################
@@ -4965,6 +4982,24 @@
 
 ;owl_escape_value(A) > 0 || owl_escape_value(B) > 0
 
+
+Pattern A1504
+
+OXXXO         threaten to take an eye by ko
+.OX.X
+*..X.
+-----
+
+:8,-,value(60)
+
+OXXXa
+bcX.X
+*..X.
+-----
+
+;!attack(a) && !xplay_defend(b,c)
+
+
 #########################################################
 #                                                       #
 #               Specific edge attacks                   #
@@ -5111,14 +5146,17 @@
 :8,s,value(50)
 
 
-Pattern A1609
-
-?OOY?
-O.X.X
-OX.*.
------
-
-:8,s,value(35)
+# nn Removed (3.3.13)
+#   Success rate in regressions : 0% (0/35)
+# 
+# Pattern A1609
+# 
+# ?OOY?
+# O.X.X
+# OX.*.
+# -----
+# 
+# :8,s,value(35)
 
 
 Pattern A1610
Index: patterns/owl_defendpats.db
===================================================================
RCS file: /cvsroot/gnugo/gnugo/patterns/owl_defendpats.db,v
retrieving revision 1.77
diff -u -r1.77 owl_defendpats.db
--- patterns/owl_defendpats.db  14 Dec 2002 16:30:00 -0000      1.77
+++ patterns/owl_defendpats.db  14 Dec 2002 16:53:18 -0000
@@ -5009,12 +5009,15 @@
 
 
 Pattern D1314
+# nn Modified (3.3.13)
+#   It's rather pointless to match at the edge
 
 o.o?
 O.O.     run away
 ....
 ....
 .*..
+????
 
 :8,E,value(55)
 
@@ -5023,6 +5026,7 @@
 ....
 ....
 .*..
+????
 
 ;owl_escape_value(*) > 0
 
@@ -5747,18 +5751,38 @@
 
 
 Pattern D1356
+# nn Modified (3.3.13) see also D1356a
 
 o....         jump towards safety
 O.*..
 o....
 
-:8,E,value(80)
+:-,E,value(90)
 
 o..a.
-O.*b.
+d.*b.
 o..c.
 
-; owl_escape_value(a) + owl_escape_value(b) + owl_escape_value(c) > 0
+; (owl_escape_value(a) + owl_escape_value(b) + owl_escape_value(c) > 0)
+; && !oplay_disconnect(*,d,*)
+
+
+Pattern D1356a
+# nn New pattern (3.3.13)
+#   B attribute since cut possible
+
+o....         jump towards safety
+O.*..
+o....
+
+:-,BE,value(80)
+
+o..a.
+d.*b.
+o..c.
+
+; (owl_escape_value(a) + owl_escape_value(b) + owl_escape_value(c) > 0)
+; && oplay_disconnect(*,d,*)
 
 
 Pattern D1357