Re: [Vrs-development] Who's doing the distributed filesystem?

[Eric Altendorf]

On Monday 29 April 2002 05:52, Bill Lance wrote:
> That's my thought here also, Eric.  It's basically a
> type 10 RAID system with encrypted blocks and spread
> accross the network

The difference being that it can take seconds for the disks to talk to each 
other, disks can go on and offline at will, disks could theoretically 
masquerade under false identities, and files need to be distributed and 
fragmented and mirrored in rather particular ways.  It all makes it very 
difficult to guarantee the ACID qualities of file operations...not 
impossible, just difficult.

> If this is the area that you would like to focus on,
> that would be great.  We are taking a top-down design
> approach.  Where we are at so far is in the Docs in
> the Repositiry Manager section of both the 'Local Data
> Server Design Detail' page and the Transactions page.

OK.  So, first of all, the document makes no mention of the desired file 
semantics...that's probably the first thing one needs to know before 
designing a filesystem. :-)

Do we want Unix semantics (writes and reads are serialized so the
effects of a write are immediately visible)?  This requires that when a
write is issued on a particular virtual block, all LDS nodes hosting
that block must update the block's value and confirm, probably using a
two phase commit protocol, before the write can be deemed "complete".
Not only will this make writes very slow, but it will make reads from
that same filesystem block wait until the write is complete, thus
slowing down the filesystem as a whole.  We should be able to make reads
pretty fast with caching, but only at the expense of making writes
potentially very expensive.  Is this OK?


Also, I am still unclear on how the services interact with the filesystem.  
It's not a kernel-level filesystem, so it will not be transparent to users.  
How will we make arbitrary services (eg Apache) read and write on the virtual 
filesystem?


Eric

-- 
"First they ignore you.  Then they laugh at you.
 Then they fight you.  And then you win."             -Gandhi