Re: Interesting question/experiment about value of cube ownership

[MK]

On 3/1/2024 6:22 AM, Ian Shaw wrote:

> 27000 trials at 0-ply and 1-ply. 135000 trials at 2-ply.
> There’s almost no difference in value between the rollout
> that took 8 minutes and the one that tool 23 hours, which
> speaks to the strength of the initial evaluation.

This is good to know. Can you post the position ID so that
there is no misassumptions.

> The rollout suggests that the value of cube ownership in
> the initial position is worth about 0.36 points.

This is very interesting. Is making the bot auto-play the
same as doing rollouts? During the past weeks, I have done
12 different experiments with 20,000 games in each. I'm now
putting it all on a neatly organized web page which I will
share here soon.

Six of my experiments were about the value of winning the
opening roll and/or owning the cube from the start (i.e.
before the first move for the mutant but before the second
move for the bot since it always auto-plays and there is no
way to intercept before its first move).

Very interestingly I also came up with 0.36 ppg and 0.28 ppc
("points per cube" decision).

I collected and tabulated quite a lot of various stats which
will be on my web page, along with the actual scripts I ran,
saved games, log files, etc. so that you all can derive your
own conclusions with or without replicating my experiments,
with the important ones being about "mutant cube strategies".

> One thing to notice is that the rollout has the on-roll
> player winning about 1% less than the evaluations posted
> by MK. I think this is due to the evaluation assuming that
> initial doubles may be played, whereas I set the rollout
> to play as the initial position.

I'm not sure what you are referring to here. What I had posted
was the GnuBG's 2-ply evaluation of the opening position (i.e.
without initial doubles). So, that 1% must be the difference
between that and your rollouts?? (as well as my experiments?)

> I haven’t found a way toa ask gnubg for an evaluation for the
> initial roll. Is there one?
> You could get a 1-ply evaluation by combining all 15 0-ply
> evaluations of the first roll, and so forth.

I don't understand these. Hopefully others will pitch in their
comments in response...

MK


> *From:*bug-gnubg-bounces+ian.shaw=riverauto.co.uk@gnu.org 
> <bug-gnubg-bounces+ian.shaw=riverauto.co.uk@gnu.org> *On Behalf Of *Ian Shaw via Bug reports for and 
> general discussion about GNU Backgammon.
> *Sent:* Thursday, February 8, 2024 11:39 AM
> *To:* playbg-rgb@yahoo.com; bug-gnubg@gnu.org
> *Cc:* Philippe Michel <philippe.michel7@free.fr>
> *Subject:* RE: Interesting question/experiment about value of cube ownership
>
> It just so happens that I rolled out the opening position a few days ago for another reason. This 
> was at 7-away 7-away rather than $ play, because I was interested in match play. I doubt that makes 
> a huge difference.
>
> This was using gnubg-1_08_dev-20240103-setup.exe not the newest gnubg-1_08_001-20240204-setup.exe 
> that Philippe released recently.
>
> Philippe, am I correct in thinking that the cube handling on these two versions is the same? Your 
> announcement emails both include the same comment.
>
> “Improvement to cube decisions at 0- and 1-ply and weaker levels. Cube error rates are approximately 
> halved and the repartition of errors (premature doubles vs. missed doubles vs. take or pass errors) 
> is now similar to higher plies instead of being mostly premature doubles.”
>
> The rollout results indicate about 1% fewer wins for the roller than the 
> evaluations.
>
> 4HPwATDgc/ABMA:cAngAAAAAAAE
>
> Cube analysis
>
> Rollout cubeless equity +0.0408 (Money: +0.0396)
>
> Cubeful equities:
>
> 1. No double           +0.0655
>
> 2. Double, pass        +1.0000  (+0.9345)
>
> 3. Double, take        -0.2999  (-0.3654)
>
> Proper cube action: No double, take (28.1%)
>
> Rollout details:
>
> Centered 1-cube:
>
>    0.5129 0.1480 0.0083 - 0.4871 0.1351 0.0073 CL +0.0408 CF +0.0655
>
> [0.0001 0.0002 0.0001 - 0.0001 0.0001 0.0001 CL  0.0003 CF  0.0008]
>
> gnubg owns 2-cube:
>
>    0.5156 0.1522 0.0091 - 0.4844 0.1375 0.0150 CL +0.1216 CF -0.2999
>
> [0.0001 0.0002 0.0001 - 0.0001 0.0002 0.0002 CL  0.0007 CF  0.0012]
>
> Full cubeful rollout with variance reduction
>
> 186624 games, rollout as initial position, Mersenne Twister dice generator with 
> seed 823069761
>
> Play: world class 2-ply cubeful prune [world class]
>
> keep the first 0 0-ply moves and up to 8 more moves within equity 0.16
>
> Skip pruning for 1-ply moves.
>
> Cube: 2-ply cubeful prune [world class]
>
> Cheers,
>
> Ian
>
> -----Original Message-----
> From: bug-gnubg-bounces+ian.shaw=riverauto.co.uk@gnu.org 
> <mailto:bug-gnubg-bounces+ian.shaw=riverauto.co.uk@gnu.org><bug-gnubg-bounces+ian.shaw=riverauto.co.uk@gnu.org <mailto:bug-gnubg-bounces+ian.shaw=riverauto.co.uk@gnu.org>> On Behalf Of MK
> Sent: Thursday, February 8, 2024 2:23 AM
> To: bug-gnubg@gnu.org <mailto:bug-gnubg@gnu.org>
> Subject: Interesting question/experiment about value of cube ownership
>
> I'm chugging along with my mutant cube skill experiments as I can spare time, saving all games, 
> which I will share on my web site, when I'm done, along with my scripts.
>
> While doing the double at > 50% experiment, I remembered an old question I had asked in RGB about a 
> year ago: What if the winner of the opening roll is allowed pre-double?
>
> See thread:
>
> https://groups.google.com/g/rec.games.backgammon/c/BVEnaqGM6dg/m/2c685q4DAAAJ 
> <https://groups.google.com/g/rec.games.backgammon/c/BVEnaqGM6dg/m/2c685q4DAAAJ>
>
> When you evaluate the opening position in GnuBG, this is what you get:
>
> =========================================================
>
> Position ID:         4HPwATDgc/ABMA
>
> Match ID:            cAkAAAAAAAAA
>
> Evaluator:            Contact
>
>           Win     W(g)    W(bg)   L(g)    L(bg)   Equity    Cubeful
>
> static:  52.1    15.4     0.8    13.0     0.8   +0.067    +0.084
>
>    1 ply:  52.7    14.8     0.9    12.9     0.5   +0.076    +0.098
>
>    2 ply:  52.5    14.9     0.7    12.5     0.5   +0.076    +0.099
>
> Cube analysis
>
> 2-ply cubeless equity +0.076
>
>      52.5  14.9   0.7 -  47.5  12.5   0.5
>
> Cubeful equities:
>
> 1. No double           +0.099
>
> 2. Double, pass        +1.000  (+0.901)
>
> 3. Double, take        -0.171  (-0.270)
>
> Proper cube action: No double, take (23.0%) 
> =========================================================
>
> I have created a Python script to intervene if the human player wins the opening roll, to set the 
> cube at 2 owned by the bot, and then to execute "end game" command, for the bot to play for both 
> sides at the same checker and cube skill settings.
>
> So, you know the equity gained by winning the opening roll and the equity lost by making the cube 
> error at the same time, before the first move. Can anyone tell me what I will be expecting to see 
> after, let's say,
>
> 10,000 games, in terms of which side will win/lose by what percentage?
>
> BTW: I already know. ;) I'm asking to see how confident are you in GnuBG's equity and/or error 
> calculations and how competent are you to make mathematical predictions?
>
> MK