RE: [Bug-gnubg] Gnu 2ply seems too optimistic

[Ian Shaw]

Mochy wrote on 06 November 2006 01:38

> I use:
> Version 0.14-mingw (build Oct 11 2006)
> Snowie 2.1 Table
> Evaluations settings are attached.
> 
> On Sun, 5 Nov 2006 15:22:25 +0100
> "Christian Anthon" <address@hidden> wrote:
> 
> > Which build do you have, which MET, and details on the 
> gnubg evals please.
> > 
> > Christian.
> > 
> > On 11/5/06, Mochy <address@hidden> wrote:
> > > Recently I found 3 positions where Gnu 2ply evaluate the 
> equity too 
> > > optimistic.After the rollout or in Snowie evaluation (and
> > > Rollout) , Those are actually huge pass.
> > > Not only 2ply, 4ply also make similar mistakes.
> > >
> > > I dont know if it is because of new build or not.
> > > Any comments are welcome.
> > >
> > > Here are 3 positions (Position ID / Match ID) ZtvAAWBjN4MIDA / 
> > > cAmgAQAAAAAA bFsIOEhxOwcDIA / MAGgAVAAMAAA xs7BAURwb4OFAA / 
> > > MAHgADAAAAAA
> > >

There are many positions that gnubg gets wrong. For example, it is well
known that gnubg overestimates the equity for the side playing a holding
game, leading to some bad takes. It won't be anything to do with the new
build. The evaluation engine has not changed for some time; all the
recent changes have been to the user interface, or to make speed
improvements.

As far as I know, nobody is working on improving gnubg's evaluations at
the moment. Rollouts will be essential for the foreseeable future if you
want to check on gnubg's evaluations. 

None of your three positions look unusual to me, so it is unsurprising
that there are mistakes. However, I get some different results to you
(using the g11 MET). 

The first position, gnubg reports as No Double/Take, as you state.
However, position 2 IS evaluated as a pass at 2-ply. Four-ply thinks
it's a take, and 3-ply has it as too good.

 GNU Backgammon  Position ID: bFsIOEhxOwcDIA
                 Match ID   : MAGgAVAAMAAA
 +12-11-10--9--8--7-------6--5--4--3--2--1-+     O: White
 |             O  X |   | O  O  O     X  O |     5 points
 |             O  X |   | O  O  O          |     On roll
 |             O  X |   | O     O          |     
 |                  |   |                  |     
 |                  |   |                  |    
^|                  |BAR|                  |     13 point match (Cube:
1)
 |                  |   |                  |    
 |                  |   |                  |     
 |                  |   |                  |     
 | O                |   | X  X  X  X       |     
 | O  X           X | X | X  X  X  X     O |     6 points
 +13-14-15-16-17-18------19-20-21-22-23-24-+     X: Blue


Cube analysis
2-ply cubeless equity  +0.615 (Money:  +0.587)
  0.616 0.422 0.033 - 0.384 0.098 0.002
Cubeful equities:
1. Double, pass         +1.000
2. Double, take         +1.058  ( +0.058)
3. No double            +0.775  ( -0.225)
Proper cube action: Double, pass


Cube analysis
3-ply cubeless equity  +0.875 (Money:  +0.840)
  0.692 0.507 0.024 - 0.308 0.073 0.003
Cubeful equities:
1. No double            +1.012
2. Double, take         +1.663  ( +0.650)
3. Double, pass         +1.000  ( -0.012)
Proper cube action: Too good to double, pass (1.9%)

Cube analysis
4-ply cubeless equity  +0.564 (Money:  +0.538)
  0.604 0.402 0.027 - 0.396 0.097 0.002
Cubeful equities:
1. Double, take         +0.949
2. Double, pass         +1.000  ( +0.051)
3. No double            +0.776  ( -0.172)
Proper cube action: Double, take

Position 3 IS evaluated as Double Pass, with both Snowie and g11 MET.

 GNU Backgammon  Position ID: xs7BAURwb4OFAA
                 Match ID   : MAHgADAAAAAA
 +12-11-10--9--8--7-------6--5--4--3--2--1-+     O: White
 | X           O  O |   | O  O     X       |     3 points
 | X           O  O |   | O  O             |     On roll
 | X                |   | O  O             |     
 |                  |   | O                |     
 |                  |   |                  |    
^|                  |BAR|                  |     7 point match (Cube: 1)
 |                  |   |                  |    
 |                  |   |                  |     
 |             X    |   | X                |     
 | O           X    |   | X  X        X    |     
 | O  O        X  O | X | X  X        X    |     0 points
 +13-14-15-16-17-18------19-20-21-22-23-24-+     X: Blue


Cube analysis
2-ply cubeless equity  +0.706 (Money:  +0.641)
  0.734 0.225 0.006 - 0.266 0.057 0.001
Cubeful equities:
1. Double, pass         +1.000
2. Double, take         +1.133  ( +0.133)
3. No double            +0.888  ( -0.112)
Proper cube action: Double, pass

Cubeful equities: (Snowie MET)
1. Double, pass         +1.000
2. Double, take         +1.221  ( +0.221)
3. No double            +0.911  ( -0.089)
Proper cube action: Double, pass

Cube analysis
4-ply cubeless equity  +0.702 (Money:  +0.641)
  0.738 0.213 0.006 - 0.262 0.053 0.001
Cubeful equities:
1. Double, pass         +1.000
2. Double, take         +1.143  ( +0.143)
3. No double            +0.895  ( -0.105)
Proper cube action: Double, pass