Re: [Bug-gnubg] Re: The importance of METs

[kvandoel]

On Mon, 8 Sep 2003, Joern Thyssen wrote:

> On Mon, Sep 08, 2003 at 10:39:37AM +0200, address@hidden wrote
> > Now that  the external player  bugs are dead  I tried playing  some 17pt
> > matches expert-expert using different met's.
> >
> > I  think the  results show  that the  differences between  the  mets are
> > completely negligible in terms of equivalent rating strength points.
> >
> > Playing 100 17pt matches  mec26-woolsey, analyse using woolsey.xml (also
> > at expert  level) I get the  follolwing effective ratings  for the mec26
> > table with 2200 being defined as expert level. On average the difference
> > is a single rating point caused  by cube "errors".  Of course the expert
> > level analysis  only shows how  small the difference between  the tables
> > is.
>
> I think you need much more than 100 matches.

It depends on your goals. I don't intend what you describe in the
following paragraph, just to show that the actual diffrerence in play
are very tiny.

> If mec26 was a 51-49 favorite you would need around 20,000 to get a
> 95% CI interval of +/- 1.96 * (sqrt(0.5/20,000)) = +/- 1%.
>
> If mec26 is a 50.1 to 49.9 favourite then you need 2,000,000 matches :-)
>
> Tricks like pairing dice may reduce the variance leading to fewer
> games, and you may use the luck adjusted results to reduce the variance.
>
>
> It would be more interesting to do the experiment at various match
> scores instead. I guess that was what Douglas suggested?
>
> This should detect the match scores where each table has problems. Of
> course, if the table is wrong at, say, -3,-4, this will cause ripple
> effects through the table.

Someone else will have to do this.

Kees