Re: [Sks-devel] Out of the pool

[Phil Pennock]

On 2018-01-26 at 09:48 +0100, Kiss Gabor (Bitman) wrote:
> > If enough people are sending the signal to regenerate stats every hour,
> > then the distribution of total key counts would cluster around a higher
> > value, so that people who rely solely upon daily key generation might
> > drop more than two stddevs below the mean (of numbers after outlier
> > exclusion).

> BTW. Let's assume a TLA wants to control HKP traffic of a target
> person. (Someone who is worthing some investments like Snowden or
> Assange.) A possible attack vector is this:

A few innocent servers which announce a high fake number are likely to
be dropped in the first-pass outlier exclusion, the bit which keeps
keyservers with 0 keys, or 10,000 keys, from dragging down the numbers,
and also any idiots who bump the number up far too far.

Then the mean and stddev are calculated over what's left, so "probably
reasonable" servers.  Which might include some which are too many.

But part of the point of "mean - 2*stddev" (which, from memory (not
rechecked) is the approach Kristian copied from me :) ) is that if the
spread increases because of that sort of behavior, then the stddev
increases.  Assuming a normal distribution, you'd be dropping the
smallest 2% of servers.  Assuming a non-normal distribution, you're
still only dropping a small percentage.

Thus if a server is only calculating stats once per day, then with
enough servers calculating hourly, and a high enough rate of new key
influx, it's conceivable that you'd get dropped.  It's why there's also
a fudge-factor in the cut-off limit (300 or so, when I last looked) to
account for "how many updates in the course of a day" to let most
servers stay in the pool anyway.

The idea is to exclude _broken_ keyservers, since with SKS working the
keys will all get out "soon enough" anyway.

Fixes under keyserver operator control:
 * add a cron-job to signal SKS to regenerate stats

Fixes under Kristian's control:
 * use his stats on keyserver metrics to derive a better constant
   fudge-factor to keep most servers in the pool, and add a recurring
   calendar event to double-check that number every 6 months

Looks like the fudge-factor needs to account for 1200 new keys per day,
now.

-Phil