Re: [Bug-gnubg] Re: Strange FIBS ratings

[Christopher D. Yep]

At 07:04 PM 9/8/2003 +0200, Jim Segrave wrote:
> On Mon 08 Sep 2003 (14:52 +0000), Joern Thyssen wrote:
> > On Mon, Sep 08, 2003 at 03:03:55PM +0200, Jim Segrave wrote
> > > On Mon 08 Sep 2003 (11:04 +0000), Joern Thyssen wrote:
> > > > On Mon, Sep 08, 2003 at 11:36:33AM +0200, Jim Segrave wrote
> > > >
> > > > Kees' experiments show that cube decisions errors don't weigh as 
> much as
> > > > chequer play errors. I can't offer any explanation for this, other than
> > > > gnubg's chequerplay is much better than the cube play???
> > >
> > >

I think this phenomenon has been known for many years now.  Kees' 
experiments and Douglas Zare's research on Gammonvillage are just the 
latest examples supporting this conclusion.

While the largest errors in a money game or match are likely to be cube 
errors (which causes most players to mistakenly believe that the total 
equity given up by cube errors dominates the total equity given up by 
checker errors), the total equity given up by checker errors is larger (on 
average) since there are more checker errors per game.  For example it's 
possible to make a checker error on every turn of the game, while much of 
the game one doesn't even have cube access (i.e. opponent owns the 
cube).  Even if we only consider non-trivial decisions, there are a lot 
more non-trivial checker decisions than non-trivial cube decisions per 
game.  Note also that it's only possible to make 1 wrong pass per game, a 
few wrong doubles per game (it would be rare for one player to make more 
than 1 wrong double in a given game), a few wrong takes per game (again 
it's rare to make more than 1 in a given game), and several missed doubles 
per game.

David Montgomery's excellent article, archived at 
http://www.bkgm.com/rgb/rgb.cgi?view+455, gives some general arguments as 
to why checker play is difficult.

> > > >From a comment in the thread on GammOnLine:
> > >
> > > ================
> > >
> > > seems to me that checker play errors in real matches represent are
> > > always an irretrievable loss of equity, while cube errors may or may
> > > not matter, depending on the flow of the game (5 missed marginal
> > > doubles with an eventual correct double/take), and opponent's error
> > > (too good to double, but he took.) Objective cube errors may not even
> > > be errors (there's little play-the-opponent in checker play, but a lot
> > > in cube action). Further, it seems that cube errors against weaker
> > > opponents are relatively less costly than cube errors against stronger
> > > opponents (against a weakie I can recover from a bad take and gammon
> > > loss in the first game of a 5-point match, or choose to play the whole
> > > match semi-cubeless, or take "passes" that opponent's checker errors
> > > make takes -- in gnu's eyes I'll be a "casual cubist" in all cases).
> > >
> > > ================
> >

The above is a common (but incorrect argument).  The same argument could be 
made about checker errors in repeating positions, but it's wrong.

Consider an example:

1. If a player makes a 0.1 error (checker or cube), both players then play 
perfectly, the original position later repeats (with the same dice roll if 
the original error was a checker error), and the player then makes the 
correct play (checker or cube), this doesn't mean that his original error 
shouldn't count as an error.  One way to think of it is that maybe there 
was a 50% chance that his original error would "cost" 0.2 and 50% that it 
would "cost" 0 (i.e. the position would essentially repeat).  Rather than 
base the error on the actual future dice rolls, it's better and proper to 
just give it a 0.1 error.  This is what gnubg, Snowie, etc. do.  This 
applies for both checker and cube errors.

Some specific examples:

2. Bearoff, Player 0 and Player 1 both have a 2-roll position (i.e. 3 or 4 
each on the acepoint).  Player 0 owns the cube (normalized to 1).

Suppose that player 0 fails to double.  How much of an error is this 
(answer: .278)?  The gammonline argument says that it's an error of -2.0 
(the difference between winning 1 and losing 1) 13.9% (5/6 * 1/6) of the 
time and an error of 0.0 86.1% of the time (player 0 can cash on the next 
turn 86.1% of the time if he forgets to cash on the first roll).  While 
this also averages to .278, most players don't think of equity this way 
(nor should they).

3. Bearoff, both players have 5-roll position, cube centered, player 0 on roll.

Correct cube action is marginal double, easy take.  Not doubling is a .023 
error.  The gammonline argument says that if both players roll non-doubles 
(so that player 0 can double-in next turn), then the original error wasn't 
really an error and should be counted as 0, but if other sequences happen 
then count the error as something else (including counting the error as a 
"negative error" if player 0 rolls non-doubles followed by player 1 rolling 
doubles!).  While this again averages to .023, it's the wrong way to 
consider equity.

4. Bearoff, player 0 has checkers on 1,2; player 1 has a 2-roll 
position.  Player 1 owns the cube (normalized to 1).  Player 0 on 
roll.  Player 0 rolls 6-1 and plays 2/1, 1/off instead of 2/off 
1/off.  This is an error of .333 (equity of 5/6 - 1/6 = 2/3, instead of 
1).  It's right to consider this a .333 error instead of a 0.0 error 5/6 of 
the time and a 2.0 error 1/6 of the time.

A more generalized example:

5. Player 0 has a medium to large race lead and is playing against a 
20-point holding game.  Suppose proper cube action is double/take (and is 
likely to remain so for many rolls).  Suppose player 0 (a beginner) doesn't 
know to double (and makes the same mistake on all future rolls).  Let's 
also assume that he does know to double when he clears his midpoint or 
leaves a single checker on his midpoint which is missed.  For simplicity 
assume that in all games player 0 will roll doubles to clear the midpoint 
or will leave a shot (and get hit or missed).  Suppose that overall, player 
0's late-double strategy costs .30 in equity.  The gammonline argument says 
to: (a) penalize him zero when he doesn't double then rolls a non-double; 
(b) penalize him a large amount when he doesn't double then rolls doubles 
(which is a market loser); (c) penalize him a negative amount when he 
doesn't double, leaves a number which forces him to leave a shot, and is 
hit; and (d) penalize him a large amount when he doesn't double, leaves a 
number which forces him to leave shot, and is missed (this is a market 
loser).  The (correct) approach that gnubg (and Snowie, etc.) use is to 
penalize each missed double a certain amount (e.g. .05).  In some games the 
beginner will clear his midpoint after his first missed double (for that 
game gnubg will assign him a total error of only .05), while in other games 
he may roll a large number (e.g. 12) of non-doubles before clearing his 
midpoint or leaving a shot (for that game gnubg will assign him a total 
error of .60).  However, on average the beginner will be credited with 
errors totalling .30 for this late-double strategy.

Also, the side comments about cube decisions which are errors in theory, 
but not in practice (i.e. if the opponent is a stronger/weaker player) are 
relevant, but let's ignore them for the point of this discussion.  gnubg 
with (and without) noise doesn't try to play based on the strength of the 
opponent, yet we observed that it still gives up much more in checker 
errors than cube errors.  Two equally strong humans (who know they are 
equally strong) also give up much more in checker errors than in cube 
errors.  The gammonline side comments support the idea that if a human (who 
is consciously adjusting for skill) plays a match, then analyzes it with 
gnubg, his reported cube error rate will be higher than it "should" be in 
practice.  (BTW, I agree with this statement.)

> In the case of missed doubles, you may not lose your market and you
> will get a chance to recover your mistake on the next move?

See example 3 (5-roll position).  Rather than assigning different errors 
based on future dice rolls, it's proper to just assign an error of .023 in 
the original position.

Chris