persistent DB connections (was: RE: modular database backends)

[Dirk Bergstrom]

one of my main reasons to move to a database backend is performance &
scaling; setting up & tearing down a database connection for every gnats
transaction is just not the way to make a speedy & scalable application.  i
believe that a database-backed gnats will need some way to maintain a
persistent connection to that database.

currently gnatsd is spawned by inetd as needed, and it exits after
performing a single transaction (generally).  here's a few ways we could
handle the database connection:

1) keep the current model, adding a separate, persistent, "database daemon".
each connection would spawn a new instance of gnatsd, which would connect to
the database-daemon as needed.  i happened across sqlrelay
(http://www.firstworks.com/sqlrelay.html), a GPLed package which appears to
do just what's needed for the database daemon.

2) a persistent daemon that handles setting up the connection to the client,
and holds open a connection (or several connections) to the database.  as
new requests come in, the daemon would fork off a child to handle each one,
up to some preset limit.

3) something like apache -- a master process forks several persistent
daemons, which take turns handling connections.  each child daemon holds a
database connection, and services many requests before shutting down.

#1 has the advantage of being most like the current codebase, and would thus
require the least work.  this would also best support a modular approach --
gnats could be installed with a database-daemon module, or the current
flat-file module.  on the other hand, each client connection has to go
through the overhead of reading in all the information about a database
(dbconfig, categories, responsible, gnatsd.access, etc.).  it's not a lot of
overhead, but i find it vaguely offensive to build something anew every
couple seconds...

#2 and #3 are fairly similar, as both allow for caching of the difficult
stuff.  i'm not sure which would be better -- i'd be inclined to say #2, but
i can't ignore the fact that apache works like #3, and it's a *very*
successful system.  in any case i think i like both of these more than #1,
though they require more rewriting.

what do y'all think?

--
Dirk Bergstrom              address@hidden
____________________________________________
Juniper Networks Inc.,  Engineering Web Guru
Tel: 408.745.3182          Fax: 408.745.8905