[Bug-gnubg] Truncated rollouts and evaluations give different results

[Christian Anthon]

Hi all,

I was looking for ways to test the rollout code and came up with the following:

a) Setup evaluation on ply 1, and turn off the pruning net.
b) Setup rollouts to truncate after 1ply, 36 trials, quasi-random dice
on, varredn off, 0ply cubeful play and cube(I believe that pruning
isn't active at 0ply play, but it was turned off anyway to be safe).
c) Start a new money session and roll 21.
d) evaluate the position after the 24/23 13/11 split
e) rollout the position after the 24/23 13/11 split

Now if everything work as expected d) and e) should give the same
result, and indeed they do:

Position ID:    4HPkASjgc/ABMA
Match ID:       MIENAAAAAAAA
Evaluator:      Contact
        Win     W(g)    W(bg)   L(g)    L(bg)   Equity    Cubeful
 1 ply: 0.507852 0.137046 0.005140 0.134718 0.004246  +0.018926  +0.025745

  0.507852 0.137046 0.005140 - 0.492148 0.134718 0.004246 CL
+0.018926 CF  +0.025745
 [0.007014 0.003629 0.000197 - 0.007014 0.003758 0.000291 CL
0.020000 CF   0.025748] 1r

The same goes for 2-ply evaluation compared to a rollout truncated at
2ply and running for 1296 trials

 2 ply: 0.493744 0.131802 0.004152 0.138802 0.005553  -0.020912  -0.024975

  0.493744 0.131802 0.004152 - 0.506256 0.138802 0.005553 CL
-0.020912 CF  -0.024975
 [0.001493 0.000924 0.000058 - 0.001493 0.001045 0.000080 CL
0.004525 CF   0.005778] 1r

However 3-ply eval compared to a rollout truncated at 3-ply running
for 46656 trials show minor differences:

3 ply: 0.507612 0.134868 0.004909 0.135156 0.004396  +0.015449  +0.020794

  0.507605 0.134899 0.004911 - 0.492395 0.135157 0.004397 CL
+0.015466 CF  +0.020540
 [0.000323 0.000210 0.000014 - 0.000323 0.000219 0.000015 CL
0.000980 CF   0.001298] 1r

The effect can be greater with other examples, but this was the
easiest one to report and reproduce.

There is only one situation that I know of where one should be
careful, and that is when the rolled out position is close to a
double, and that isn't the case here. Does anybody have an idea about
what is going on? The evaluation code is a bit harry, so I don't
really know where to start.

Christian.