Re: [Bug-gnubg] Re: The importance of METs

[Joern Thyssen]

On Mon, Sep 08, 2003 at 10:39:37AM +0200, address@hidden wrote
> Now that  the external player  bugs are dead  I tried playing  some 17pt
> matches expert-expert using different met's.
> 
> I  think the  results show  that the  differences between  the  mets are
> completely negligible in terms of equivalent rating strength points.
> 
> Playing 100 17pt matches  mec26-woolsey, analyse using woolsey.xml (also
> at expert  level) I get the  follolwing effective ratings  for the mec26
> table with 2200 being defined as expert level. On average the difference
> is a single rating point caused  by cube "errors".  Of course the expert
> level analysis  only shows how  small the difference between  the tables
> is.

I think you need much more than 100 matches.

If mec26 was a 51-49 favorite you would need around 20,000 to get a
95% CI interval of +/- 1.96 * (sqrt(0.5/20,000)) = +/- 1%.

If mec26 is a 50.1 to 49.9 favourite then you need 2,000,000 matches :-)

Tricks like pairing dice may reduce the variance leading to fewer
games, and you may use the luck adjusted results to reduce the variance.


It would be more interesting to do the experiment at various match
scores instead. I guess that was what Douglas suggested?

This should detect the match scores where each table has problems. Of
course, if the table is wrong at, say, -3,-4, this will cause ripple
effects through the table.

Jørn

pgpEq2WBbsKvb.pgp
Description: PGP signature