Re: [Sks-devel] IPv6 peering; keydumps annoyingly large

[Matthew Palmer]

On Tue, May 31, 2011 at 05:31:00PM -0700, oakwhiz wrote:
> I'd also like to take the opportunity to express my opinion on the way
> SKS stores keys.
> Seeing as the keydump is about 4GB, and continually growing, I'm
> wondering what will happen in the future as it increases.
> It will eventually become larger than a standard DVD, making it more
> difficult to transport via 'sneakernet' (physical media.)

Compress it and you should delay the inevitable until Blu-Ray burners are
standard -- or just split your dumps over two (or more) DVDs.  Or use an 8GB
USB stick (what are they now, $10?).  Or just transfer the dumps over the
Internet like most people do.  If you lack sufficient bandwidth or data
transfer cap to do that, you probably don't have enough to host a reliable
keyserver, either.

> SKS is currently the only viable keyserver in my opinion, I find it a
> bit strange that every peer must have a redundant copy of every key.
> I would prefer if the reconciliation protocol allowed keyservers to
> store only a small subset of all keys (i.e. 500MB) in a peer-to-peer
> sort of arrangement (similar to Gnutella, Freenet or BitTorrent.)

But then how would you contact pool.sks-keyservers.net and reliably get a
key?  All clients would need to support referrals (good luck getting *that*
rolled out universally any time soon), or just try random keyservers until
the key pops up, which would again require a change in client behaviour (and
when do you stop looking, on the assumption that the key doesn't exist?)

To avoid needing to change client behaviour, the server would have to go
fetch the key from another server itself and pass it on to the client, which
is slow and buggy and complicates the server code considerably.  To support
that (or referrals), all servers would need either a real-time discovery
mechanism for all keys (tricky to implement reliably), there'd need to be
centralised trackers (thus destroying one of the wonderful benefits of the
decentralised SKS network, and who'd run them, anyway?), or every server
would have to have a lookup table of all servers that have a copy of each
key -- and I wouldn't like to bet on how big that might get (especially if
every man+dog ran a keyserver, see below).

*Then* you've got to work out a scalable algorithm for distributing
knowledge of key locations, *and* a way to ensure that keys are sufficiently
redundant to survive the loss of a certain number of keyservers...  it all
gets ridiculously complicated very, very quickly.

So, yeah, sorry to wall-of-text your idea into the ground, but I don't think
it's particularly viable, compared to the fairly reliable and robust system
we've got now (whose only drawback is that it requires -- at most -- about
$16 worth of disk space, assuming you go to the expense of mirror RAID on
15k SAS drives).

> The size of the key database is discouraging for those of us who want
> to participate, but have low amounts of available disk space.

There have been expressions on this list recently that there are enough
keyservers in the pool.  Whilst I don't know if that's true or not (although
I'm starting on some work to try and find out), I'm fairly certain that the
world really doesn't need every single person running a keyserver.  If you
don't have enough disk space for the SKS key database, I really don't think
you have you beat yourself up about not participating, nor do you have to
work on complicated and fragile mechanisms to increase participation.

- Matt


-- 
A few minutes ago I attempted to give a flying fsck, but the best I could do
was to watch it skitter across the floor.
                -- Anthony de Boer, ASR