[gnugo-devel] Semeai

[bump]

I took a look at the breakage of the experimental semeai configure
option. 

14 PASS, 11 FAIL.

The time penalty is about 1% in the entire regressions. We can
certainly afford to enable the experimental semeai, if we think
it will make the engine stronger.

The 11 failures are serious enough to give one pause. It may
need a lot of debugging. I looked at all of these and report
what I found below.

When a dragon is involved in more than one semeai, results are
not dependable. But excluding this case, maybe these bugs can
be fixed. I think it is a possible that if we fix some of
these bugs during January we can consider enabling the module
for 3.4.

Breakage of 3.3.14 pre 1, experimental semeai.

./regress.sh . ld_owl.tst 
161 unexpected PASS!
./regress.sh . strategy.tst 
50 unexpected FAIL: Correct 'Q9', got 'Q11'
./regress.sh . neurogo.tst 
11 unexpected FAIL: Correct 'N5', got 'R12'
./regress.sh . viking.tst 
6 unexpected PASS!
./regress.sh . lazarus.tst 
7 unexpected FAIL: Correct 'T5', got 'L12'
./regress.sh . trevorb.tst 
120 unexpected PASS!
./regress.sh . strategy2.tst 
54 unexpected FAIL: Correct 'P6', got 'F9'
72 unexpected PASS!
80 unexpected FAIL: Correct 'P4|Q4|Q3', got 'H13'
./regress.sh . nicklas1.tst 
1213 unexpected PASS!
./regress.sh . nicklas2.tst 
903 unexpected FAIL: Correct 'critical (H8|F9) H8', got 'alive'
./regress.sh . nicklas5.tst 
1211 unexpected PASS!
./regress.sh . trevor.tst 
461 unexpected PASS!
./regress.sh . nngs.tst 
2040 unexpected PASS!
./regress.sh . global.tst 
3 unexpected FAIL: Correct 'H4', got 'J9'
34 unexpected FAIL: Correct 'N6', got 'S7'
./regress.sh . 13x13.tst 
36 unexpected FAIL: Correct 'C5', got 'B2'
39 unexpected PASS!
./regress.sh . strategy4.tst 
162 unexpected PASS!
179 unexpected PASS!
188 unexpected PASS!
./regress.sh . nngs3.tst 
240 unexpected FAIL: Correct 'G13|G12', got 'J12'
./regress.sh . nngs4.tst 
160 unexpected FAIL: Correct 'K8|L8', got 'J12'
270 unexpected PASS!
./regress.sh . ninestones.tst 
40 unexpected PASS!

FAILURES of 3.3.14 pre 1 with experimental semeai enabled.

------------------------------------------------

./regress.sh . strategy.tst 
50 unexpected FAIL: Correct 'Q9', got 'Q11'

This is an error in the handling of the ko.

------------------------------------------------

neurogo:11
11 unexpected FAIL: Correct 'N5', got 'R12'

GNU evaluates two semeai:

Considering semeai between R4 and N3
Result1: ALIVE DEAD T4, Result2 DEAD ALIVE.
Changing status of R4 from CRITICAL to ALIVE.
Changing safety of R4 from CRITICAL to ALIVE.
Considering semeai between M4 and N3
Result1: ALIVE_IN_SEKI ALIVE_IN_SEKI M6, Result2 DEAD ALIVE.
Changing status of M4 from CRITICAL to ALIVE.
Changing safety of M4 from CRITICAL to ALIVE.
Changing status of N3 from DEAD to ALIVE.
Changing safety of N3 from DEAD to ALIVE.

The semeai between M4 and N3 needs to be debugged. An
examination of the variation file shows wrong evaluation
of some terminal nodes.

------------------------------------------------

lazarus:7
7 unexpected FAIL: Correct 'T5', got 'L12'

Considering semeai between S9 and P10
Result1: ALIVE DEAD PASS, Result2 ALIVE DEAD.
Changing status of P10 from DEAD to CRITICAL.
Changing safety of P10 from DEAD to CRITICAL.
black (X) move L12

2After black at PASS, S9 is ALIVE, P10 is DEAD (1 nodes)
After white at T5, S9 is DEAD, P10 is ALIVE (100 nodes)

We need to investigate why D1376 does not produce a 
variation tree for B.

Given these results from owl_decide_semeai, GNU should 
still be able to determine that T5 is the move.

------------------------------------------------

strategy2:56
54 unexpected FAIL: Correct 'P6', got 'F9'

Considering semeai between P4 and Q6
Result1: ALIVE DEAD P6, Result2 DEAD ALIVE.
Changing status of P4 from CRITICAL to ALIVE.
Changing safety of P4 from CRITICAL to ALIVE.

Top moves:
1. F9  35.62
2. P6  27.93

Running with experimental semeai enabled reduces the
value of P6 to 27.93 points because P4 is no longer
critical. Q6 is still critical.

------------------------------------------------

strategy2:80
80 unexpected FAIL: Correct 'P4|Q4|Q3', got 'H13'

Considering semeai between R4 and R5
Result1: DEAD ALIVE PASS, Result2 ALIVE_IN_SEKI ALIVE_IN_SEKI.
Changing status of R4 from CRITICAL to ALIVE.
Changing safety of R4 from CRITICAL to ALIVE.
Changing status of R5 from DEAD to ALIVE.
Changing safety of R5 from DEAD to ALIVE.

After white at PASS, R4 is DEAD, R5 is ALIVE (250 nodes)
After black at Q4, R4 is ALIVE_IN_SEKI, R5 is ALIVE_IN_SEKI (250 nodes)

Q4 clearly wins the semeai unconditionally, so this semeai needs to be
debugged. At variation 222, B does not answer an atari.

------------------------------------------------

nicklas2.tst:903
903 unexpected FAIL: Correct 'critical (H8|F9) H8', got 'alive'

thrashing dragon found at E9
Considering semeai between F8 and E9
Result1: ALIVE DEAD PASS, Result2 DEAD ALIVE.
Changing status of F8 from CRITICAL to ALIVE.
Changing safety of F8 from CRITICAL to ALIVE.
Considering semeai between F4 and E9
Result1: ALIVE DEAD PASS, Result2 DEAD ALIVE.
Changing status of F4 from CRITICAL to ALIVE.
Changing safety of F4 from CRITICAL to ALIVE.

This is a hard problem involving two semeais. 

------------------------------------------------

global:3
3 unexpected FAIL: Correct 'H4', got 'J9'

Considering semeai between R12 and H9
Result1: DEAD ALIVE PASS, Result2 ALIVE ALIVE.
Changing status of R12 from CRITICAL to ALIVE.
Changing safety of R12 from CRITICAL to ALIVE.
Changing status of H9 from CRITICAL to ALIVE.
Changing safety of H9 from CRITICAL to ALIVE.

This problem might be a good place to start debugging.

The semeai module finds no variations. The owl code gets the
killing move at H4 from VA9.

------------------------------------------------

global:34
34 unexpected FAIL: Correct 'N6', got 'S7'

Considering semeai between O6 and O5
Result1: ALIVE DEAD N6, Result2 DEAD ALIVE.
Changing status of O6 from CRITICAL to ALIVE.
Changing safety of O6 from CRITICAL to ALIVE.

After white at PASS, O5 is DEAD, O6 is ALIVE (1 nodes)
After black at N6, O5 is DEAD, O6 is ALIVE (117 nodes)

This needs debugging. GNU finds the connection at N6,
but no followup moves.


------------------------------------------------

13x13:36
36 unexpected FAIL: Correct 'C5', got 'B2'

Considering semeai between C2 and E6
Result1: DEAD ALIVE PASS, Result2 ALIVE DEAD.
Changing status of B1 from CRITICAL to ALIVE.
Changing safety of B1 from CRITICAL to ALIVE.

B1 is not part of the dragon in this semeai.

finished examine_position
After white at J6, C2 is ALIVE, E6 is DEAD (6 nodes)
After black at PASS, C2 is ALIVE, E6 is DEAD (11 nodes)

The semeai is also analyzed wrong.

After white at PASS, C2 is DEAD, E6 is ALIVE (155 nodes)
After black at B2, C2 is DEAD, E6 is ALIVE (13 nodes)

------------------------------------------------

nngs3:240
160 unexpected FAIL: Correct 'K8|L8', got 'J12'

Considering semeai between H13 and H14
Result1: ALIVE DEAD H15, Result2 ALIVE ALIVE.
Considering semeai between H13 and L17
Result1: ALIVE DEAD J12, Result2 DEAD ALIVE.
Changing status of H13 from CRITICAL to ALIVE.
Changing safety of H13 from CRITICAL to ALIVE.

After black at H15, H13 is ALIVE, H14 is DEAD (1 nodes)
After white at PASS, H13 is ALIVE, H14 is ALIVE (1 nodes)

No variations are considered. Changing the status of H13 to 
ALIVE is nonsensical since it can be killed tactically.

This is another "debug-me-first" position.

------------------------------------------------

nngs4:160
160 unexpected FAIL: Correct 'K8|L8', got 'J12'

thrashing dragon found at E5
Considering semeai between K4 and E5
Result1: ALIVE DEAD PASS, Result2 DEAD ALIVE.
Considering semeai between K4 and J7
Result1: ALIVE DEAD PASS, Result2 DEAD ALIVE.
Considering semeai between K4 and M6
Result1: ALIVE DEAD PASS, Result2 DEAD ALIVE.
Changing status of K4 from CRITICAL to ALIVE.
Changing safety of K4 from CRITICAL to ALIVE.

Again, this is a position with multiple semeais,
likely to be difficult. But the K4/M6 semeai is
misunderstood.

After black at PASS, K4 is ALIVE, M6 is DEAD (1 nodes)
After white at PASS, K4 is ALIVE, M6 is DEAD (1 nodes)

No variations are considered. The reason given is "Two eyes versus none".

Dan