Re: Error rate and Luck adjusted result

[Francesco Ariis]

Murat and Ian,

    many thanks for your explanations. I feel I know and undestand
the topic much better now, and conversely that some players —
even strong players — at my local circle could benefit from this
thread.

I am reading your “MK vs. bot” experiments Murat and they are
extremely interesting. I believe there is much value in empirical
test like yours

—F

Il 06 giugno 2024 alle 21:40 Murat K ha scritto:
> On 6/6/2024 1:32 PM, Francesco Ariis wrote:
> 
> > Il 06 giugno 2024 alle 11:10 Ian Shaw ha scritto:
> 
> > > .... a lucky 66 makes a much bigger difference to the overall result
> > > of the match if comes in a bearoff at double match point rather in
> > > game 1 of an 11 pointer.
> 
> > Now that I understand it, I think people online fixate too much on PR
> 
> Yes and it's much worse than that because all the "giants"
> rankings, etc. are based on PR.
> 
> What Ian says about luck also applies to "errors" because
> errors made early in games/matches don't make as much of a
> difference to the overall result as the ones made later on.
> 
> In addition, human players don't have control of "luck" but
> they have control of "errors". They may knowingly make "PR
> sacrificing decisions" even if they accept bot's calculation
> of errors as accurate. But, in fact, such decisions may not
> be errors at all and actually be the better decisions since
> there has never been any "scientifically acceptable proof"
> that the bot's decisions are always the best.
> 
> In 2019, I have done an experiment playing a session of 100
> money games against XGR+ by making as my first moves what
> the bot considered the absolute worse moves. You can see it
> at: https://montanaonline.net/backgammon/xg.php
> 
> I played against XGR+ instead of XGR++ in my other experiments
> in order to keep the real-time screen-capture videos shorter,
> which you can view at:
> 
> https://m.youtube.com/watch?v=CPhBQl5ttmo&feature=youtu.be
> https://m.youtube.com/watch?v=Nj4PKrOxxy4&feature=youtu.be
> https://m.youtube.com/watch?v=3SX136_Gs30&feature=youtu.be
> https://m.youtube.com/watch?v=McGTz8nBl_c&feature=youtu.be
> 
> Before you look at the results, try to guess for a minute how
> much such huge errors would effect the results...?
> 
> As far as number of games, I only won 26 vs bot's 74 because
> after my opening blunders the bot often doubled immediately
> and I had to drop. But as far as points, I still managed to
> win 77 points more than expected from my PR.
> 
> So, yes, "PR" is "PS"! (No typo ;)
> 
> MK
> 

Il 07 giugno 2024 alle 08:36 Ian Shaw ha scritto:
> H Francesco. 
> 
> XG PR, Snowie Error Rate, and GnuBg Error Rate are all based on measuring the 
> normalised error, although they have slightly different implementations. 
> 
> Snowie: (Your total errors) / (number of moves by both sides)
> 
> Gnubg: (Your total errors) / (number of your unforced moves)
> Because gnubg uses only one side's moves in the denominator, gnubg error 
> rates will be about double the Snowie error rate.
> 
> When XG was written, the creator thought that Grub's implementation made more 
> sense, but he wanted to have error values that were broadly equivalent to 
> Snowie, because it was what most people were familiar with. So, PR is a 
> compromise:
> PR: (Your total errors) / (number of your unforced moves) * 2
> The *2 multiplier scales gnubg error rate into values that are recognisable 
> to Snowie users.
> 
> People fixate on PR because minimising errors is the surest way to improve 
> your game. However, there may be situations where you want to take your 
> opponent into account, leading you to choose a different option. It's very 
> easy to over-adjust, though, because you are relying on your opponent to make 
> enough mistake to overcome the equity you have deliberately sacrificed. 
> 
> Regards,
> Ian
> 
> 
> -----Original Message-----
> From: Francesco Ariis <fa-ml@ariis.it> 
> Sent: Thursday, June 6, 2024 8:33 PM
> To: Ian Shaw <Ian.Shaw@riverauto.co.uk>
> Cc: bug-gnubg@gnu.org
> Subject: Re: Error rate and Luck adjusted result
> 
> Hello Ian,
> 
> Il 06 giugno 2024 alle 11:10 Ian Shaw ha scritto:
> > Error rates are "normalised". That is,  they are adjusted to make them 
> > all match the size that the error would be if you were playing a money 
> > game and the cube were at 1.
> > 
> > Luck takes into account the match score. For example, a lucky 66 makes 
> > a much bigger difference to the overall result of the match if comes 
> > in a bearoff at double match point rather in game 1 of an 11 pointer.
> 
> many many thanks, this gives me a much clearer picture, I did not know about 
> normalisation.
> 
> Now that I understand it, I think people online fixate too much on PR (which 
> I believe it is similar to what gnubg outputs as “Snowie error rate”).
> 
> Thanks again
> —F