[Qemu-devel] Re: [ANNOUNCE] Sheepdog: Distributed Storage System for KVM

[FUJITA Tomonori]

On Fri, 23 Oct 2009 09:14:29 -0500
Javier Guerra <address@hidden> wrote:

> > I think that the major difference between sheepdog and cluster file
> > systems such as Google File system, pNFS, etc is the interface between
> > clients and a storage system.
> 
> note that GFS is "Global File System" (written by Sistina (the same
> folks from LVM) and bought by RedHat).  Google Filesystem is a
> different thing, and ironically the client/storage interface is a
> little more like sheepdog and unlike a regular cluster filesystem.

Hmm, Avi referred to Global File System? I wasn't sure. 'GFS' is
ambiguous. Anyway, Global File System is a SAN file system. It's
a completely different architecture from Sheepdog.


> > Sheepdog uses consistent hashing to decide where objects store; I/O
> > load is balanced across the nodes. When a new node is added or the
> > existing node is removed, the hash table changes and the data
> > automatically and transparently are moved over nodes.
> >
> > We plan to implement a mechanism to distribute the data not randomly
> > but intelligently; we could use machine load, the locations of VMs, etc.
> 
> i don't have much hands-on experience on consistent hashing; but it
> sounds reasonable to make each node's ring segment proportional to its
> storage capacity.

Yeah, that's one of the techniques, I think.


>  dynamic load balancing seems a tougher nut to
> crack, especially while keeping all clients mapping consistent.

There are some techniques to do that.

We think that there are some existing techniques to distribute data
intelligently. We just have not analyzed the options.


> i'd just want to add my '+1 votes' on both getting rid of JVM
> dependency and using block devices (usually LVM) instead of ext3/btrfs

LVM doesn't fit for our requirement nicely. What we need is updating
some objects in an atomic way. We can implement that for ourselves but
we prefer to keep our code simple by using the existing mechanism.