Re: [Sks-devel] new keydump source

[John Clizbe]

Jeffrey Johnson wrote:
> 
> On Aug 17, 2012, at 2:31 PM, Jason Harris <address@hidden> wrote:
> 
>> Have any of the OCaml hackers ever thought about dumping keys in
>> keyid (v. SKS hash) order?  That would make less deltas when using
>> rsync.  Each keydump file would then have a set range of keys, e.g.,
>> sks-dump-0000.pgp would always have (keyids) 0x00000000 - 0x000008ff.
> 
> While I'm not an OCaml hacker, the easiest way to achieve
> key ordering would be to switch from DB_HASH -> DB_BTREE
> so that the keys are pre-sorted when preparing the dump.

I don't think a /deep/understanding of Ocaml is necessary to get this part
from the trunk, keydb.ml[0] starting at line 347:
->           let openflags = (if settings.withtxn then [Db.CREATE; 
Db.AUTO_COMMIT]
->                            else [Db.CREATE])
->           in
->           let key = kdbopen ~dbenv key_db_name Db.BTREE ~moreflags:[]
->              settings.pagesize openflags 0o600
->           in
->           let keyid = kdbopen ~dbenv keyid_db_name Db.BTREE
->              ~moreflags:[Db.DUPSORT] settings.keyid_pagesize
->               openflags 0o600
->           in
->           let meta = kdbopen  ~dbenv meta_db_name Db.BTREE
->              ~moreflags:[] settings.meta_pagesize openflags 0o600
->           in
->           let subkey_keyid = kdbopen ~dbenv subkey_keyid_db_name Db.BTREE
->              ~moreflags:[Db.DUPSORT] settings.subkeyid_pagesize
->               openflags 0o600
->           in
->           let time = kdbopen ~dbenv time_db_name Db.BTREE
->              ~moreflags:[Db.DUPSORT] settings.time_pagesize openflags 0o600
->           in
->           let tqueue = kdbopen  ~dbenv tqueue_db_name Db.BTREE ~moreflags:[]
->              settings.tqueue_pagesize openflags 0o600
->           in
->           let word = kdbopen ~dbenv word_db_name Db.BTREE
->              ~moreflags:[Db.DUPSORT] settings.word_pagesize openflags 0o600

As you see, all the files in KDB are created/opened as BTrees. However, I need
to check which value is the sort key of the table key, keyID or hash.

> The one _ESSENTIAL_ additional change might be to override
> the key comparison with memcmp (the default in Berkeley DB) if
> little endian integers were somehow used as keys.

I believe that's handled by the Ocaml runtime. I'll check after dinner

-John

[0] https://bitbucket.org/skskeyserver/sks-keyserver/src/ac4a7159ca28/keydb.ml

-- 
John P. Clizbe                      Inet: John (a) Gingerbear DAWT net
SKS/Enigmail/PGP-EKP                  or: John ( @ ) Enigmail DAWT net
FSF Assoc #995 / FSFE Fellow #1797  hkp://keyserver.gingerbear.net  or
     mailto:address@hidden

Q:"Just how do the residents of Haiku, Hawai'i hold conversations?"
A:"An odd melody / island voices on the winds / surplus of vowels"

signature.asc
Description: OpenPGP digital signature