Re: Skill level names

[Francesco Ariis]

Hello Timothy,

Il 08 luglio 2024 alle 15:45 Timothy Y. Chow ha scritto:
> But now think about what happens if we delete forced moves from the
> denominator. That means that errors occurring in games with a lot of forced
> moves hurt our PR more than errors occurring in games with no forced moves.
> In two separate games, I might make a error of exactly the same size, but in
> one game I get unlucky and get closed out. My PR will probably suffer more
> in the game where I have bad luck, because I'll be dividing my equity loss
> by a smaller number. Is this what we really want from PR? Maybe, maybe not.
> It's not obvious to me. A large majority of the backgammon community has
> somehow gone along with this way of doing things without thinking it
> through, or even recognizing that there is something to think about here.

I will keep it short because this is getting off-topic.
I think the problem you illustrate is a classic example of a sampling
confidence.

GNU Backgammon Error rate, as described by Ian, is a very sensible
indicator.  It is also to immediately more informative than either
Snowie ER or XG PR: “With every unforced move, you lost this amount
of equity, expressed in thousandths”.

You are correct in noticing that — as with every estimator — when the
number of samples is very low (in your example, we get closed out after
a few moves), the resulting estimate will be not reliable.

This can be highlighted in various ways.
The best option would be putting confidence intervals around the estimate.
It is the most precise way to do it, but maybe this information will not
be clear to all players.
A simpler and way more accessible option is to just print the number of
unforced moves.  Despite not being precise, everyone can understand that
a PR calculated on two unforced moves is much less meaningful than one
calculated on a hundred moves.

Happy rolls
—F