Re: [Chicken-users] Enterprise and scalability

[F. Wittenberger]

Am Sonntag, den 25.01.2009, 14:59 +0000 schrieb Alaric Snell-Pym:
> > There are a few more minor issues related to higher load, which I  
> > shall
> > post about (the one I currently have in my mind: beware of garbage
> > collection between fork and exec if finalizers might come into play,
> > since those might fail in the child [[for instance because they access
> > file descriptors close just before the fork]] and hence the gc will  
> > loop
> > forever) as time permits.
> 
> That is an interesting one! I can see a few solutions.
> 
> 1) Force a gc before the fork - but that blocks up the parent process  
> unnecessarily, and is only a stochastic fix; there's no real guarantee  
> you might not have a small nursery and end up doing another gc in the  
> child anyway.

That's what I did with chicken as the quick solution to get around.

> 2) Do a combined fork-and-exec, which is fine if there's no pre-exec  
> setup you want to do in the child (setuids, environment variables,  
> that sort of thing).

That's the solution Donovan Kolbly did for RScheme.

> 3) Have the finalizers written to deal with this situation, ranging  
> from checking the pid before cleaning up resources known to be pid- 
> local to some way of, as part of the fork, filter out finalisers that  
> shouldn't happen in the child.

Or at least run finalizers in their own exception handler.  But that
might not be enough, I'm unsure.

> I need to read more about askemos - my day job is writing a replicated  
> database (in C, alas). We have a local database on each node, which  
> many processes can read but only one writes;

So it's a typical master-slave setup.  That's going to faster on writes,
since the master can always run ahead.

But from a security point of view, it is as vulnerable as the master is.

That's why Askemos accepts the overhead of byzantine synchronisation.
It's intended for business cases, where bilateral distrust is the best
assumption over time.  Let's say closing contracts electronically,
inter-company accounting etc.

However it might be as useful for those, who want to run their website
from multiple virtual hosts rented from whatever untrusted providers
and enjoy just a kind of integrated backup and fail over for the master
host as the only benefit.

> (We use spread to handle the reliable multicasting for us)

When I came about spread, I considered a switch.  But there where
several road blocks (which where at least half way solved for Askemos at
that time): how to handle joins of new hosts (to my knowledge spread
will require some reconfiguration), privacy and authentification (secure
spread was not freely available at that time [dunno about now] while we
ran over SSL).

/Jörg