Re: R?f. : Re: JSD question [WAS : [Bug-gnubg] GNU Question]

[Jim Segrave]

On Tue 02 Sep 2003 (16:19 +0200), address@hidden wrote:
> 
> 
> As an example, imagine that in the hypothetical game tree you start from
> your initial position and you consider only two branches (the 2 moves you
> want to rollout). It may happen that the two sub-trees are not disjoint
> in early stages and in this case, there may be an important correlation
> between the two equity ditributions.
> (Note that the trees will always share something, at least in a full rollout
> at the end, when bearing off, but this may be numerically irrelevant).

I'm not sure if this isn't a different problem you're
considering. Flaws in gnubg's evaluator leading it to prefer one style
of game over another - say avoiding backgames or over-agressive
hitting will affect both game trees, that's true. But the fact that
both rollouts are wrong in the broader sense might not be relevant -
rollouts aren't producing true equity, they produce the results of
playing a position as gnubg would play it when playing against
itself. Which could be wrong.


> 
> >> But I'm not sure this independency assumption is reasonable : I would
> >> compute the estimate of the stderr of the difference of the two equity
> >> by it's definition (just like you compute the estimates of the stderr
> >> of the equities, but on the equity difference). The estimate would then
> >> include eventual correlation terms and would be equivalent (for a number
> >> of samples large enough) to the computation whenever the distributions
> >> were really independent.
> >
> >Are you suggesting treating a rollout of two different moves for n trials
> >as n pairs of results (or n occurrences of a value for the equity
> >difference) which would then be treated as an evaluation on it's own?
> 
> Yes : n occurrences of the equity difference, sum the squares, divide by n
> and take the square root (classical stderr estimator).
> 
> >This could be done, although for a rollout of more than two options,
> >you'd want to be keeping the equity differences for every pair of
> >options, as you don't know which one will be the 'best' which will be
> >used as the norm for all the comparisions (few people will care about
> >the relative ranking of the 3rd and 4th best plays, only the ranking
> >vs. the best play).
> 
> Right.
> 
> >> - eventually (if anyone else shares my doubts), run some test
> >
> >I think it's not to hard to do the above (assuming that's what you're
> >suggesting) the hardest part is probably finding a way to display the
> >results.
> 
> Why ? Just show the new "estimated jsd" (e-jsd) between the currently #1
> move and all the others, just like what it is done for the currently
> imlemented "assuming independancy jsd" (ai-jsd).
> 
> At the end, you'll just need to compare the M-1 ai-jsd with the M-1
> e-jsd (for a rollout of M moves) and see if they seem to agree or not.

Which do we display and where does one display the differences between
the two methods (for a test setup it's no big deal, it can all end up
in the message window)

> P.S.
> I may be wrong about all this (I haven't been playing with this stat stuff
> for a while), so I would wait for confirmation before any software effort
> (I've seen the CC to bug-gnubg but I haven't received anything from it
> this afternoon).

List activity goes up and down depending on interest, knowledge,
whether or not people are feeling argumentative, the weather in
various parts of the globe and who knows what all else :-)

I have to finish cleaning up some details with regard to how rollout
extensions are handled before I'd be looking at changing the
j.s.d. calculations anyway.

-- 
Jim Segrave           address@hidden