Re: [Bug-gnubg] An old bug and possible neural nets input change

[Øystein Schønning-Johansen]

Sorry the late answer, Philippe!

This is a really interesting find. I totally agree with you. The code that calculates PIPLOSS is really unreadable and un-maintainable. Some years ago I wrote a structure to hold the board state, and I then also had to adapt the input calculation to this new board data structure. I stared at this code for a long long time, but I gave up. It was too complex. For PIPLOSS, P1 and P2 (and TIMING) I more or less left the code without the potential speedups the board data structure could give me. However, I did not see this "bug". And indeed, as you say, this is by far the most expensive neural net input to calculate. 

However, it is slow to calculate, but it seems like it is also one of the most important inputs. I did a boruta feature importance analysis of the 50 handcrafted inputs, and at the absolute top spot of feature importance PIPLOSS shines out.

Look at this figure: https://drive.google.com/file/d/1CscetLk2b8OLrlpOKzV2rCrXLNwex6EF/view?usp=sharing

So... I think maybe we have to keep the input, but maybe we can recalc it the correct value, and then retrain. Further improvements to the neural networks should include some feature engineering. I think there is some potential.

Thanks for the interesting find!

-Øystein

On Wed, May 15, 2019 at 7:16 PM Philippe Michel <address@hidden> wrote:

One of gnubg's inputs is the following quantity :

    /* Average number of pips opponent loses from hits.
     *
     * Some heuristics are required to estimate it, since we have no idea what
     * the best move actually is.
     *
     * 1. If board is weak (less than 3 anchors), don't consider hitting on
     * points 22 and 23.
     * 2. Don't break anchors inside home to hit.
     */
    I_PIPLOSS,

The wording is a bit strange (are the comment and the definition from
some old article by Berliner or something like that ?) It is points
that are meant, not anchors, and the numbering is from gnubg's internal
representation and the deep points above are the usual 23 and 24
points.

Anyway, the heuristics seem reasonable, but the code a bit later is :

    /* Piploss */

    nBoard = 0;
    for (i = 0; i < 6; i++)
        if (anBoard[i])
            nBoard++;

    [...]

    /* for every point we'd consider hitting a blot on, */

    for (i = (nBoard > 2) ? 23 : 21; i >= 0; i--)
    [...]

nBoard is not the number of made points, it counts the merely slotted
ones as well... And it has been the case since gnubg's source are
under the current source control system, for at least 19 years...


Even if the error looks gross, this is not too bad in practice :
- the error is present in both gnubg and the training tool so the
  weird resulting heuristic is used consistently
- the number of pips involved is only 1 or 2 when outfield blots count
  for ten times that
- the patterns where the condition leads to a difference are
  uncommon except (2 points + 1 slot)

Still, using "if( anBoard[i] >= 2) nBoard++;", implementing the
heuristics as described and retraining the contact network a little
seem to bring a small improvement as measured by the training tool's
benchmark.


On the other hand, this input is the most expensive one to compute by
far ; more than all the other combined. On my computer, skipping it
raises the calibration result for one thread from 130k pos/s to 250k.

Its coding is rather complicated. It could probably be improved somewhat
but, since some retraining of the contact and crashed networks would be
needed anyway, it may be more promising to use instead one or a few
other similar metrics if they were much faster to compute.

_______________________________________________
Bug-gnubg mailing list
address@hidden
https://lists.gnu.org/mailman/listinfo/bug-gnubg