Cluster Filesystem for FreeBSD - any interest?
Eric Anderson
anderson at centtech.com
Wed Jun 22 12:09:42 GMT 2005
Allan Fields wrote:
> On Tue, Jun 21, 2005 at 09:35:56AM -0500, Eric Anderson wrote:
>
>>This is something I've brought up before on other lists, but I'm curious
>>if anyone is interested in developing a BSD licensed clustered
>>filesystem for FreeBSD (and anyone else)?
>
>
> A few questions:
>
> Could this be done as a stackable file system (vnode layer distributed
> file system) or did you have something else in mind (i.e. specifically
> a full implementation of a network filesystem including storage
> layer)?
Hmm. I'm not sure if it can or not. I'll try to explain what I'm
dreaming of. I currently have about 1000 clients needing access to the
same pools of data (read/write) all the time. The data changes
constantly. There is a lot of this data. We use NFS currently.
FreeBSD is *very* fast and stable at serving NFS data. The problem is,
that even though it is very fast and stable, I still cannot pump out
enough bits fast enough with one machine, and if that one machine fails
(hardware problems, etc), then all my machines are hung waiting for me
to bring it back online.
So, what I would love to have, is this kind of setup: shared media
storage (fibre channel SAN, iscsi, or something like ggated possibly),
connected up to a cluster of hosts running FreeBSD. Each FreeBSD server
has access to the logical disks, same partitions, and can mount them all
r/w. Now, I can kind of do this now, however there are obviously some
issues with this currently. I want all machines in this cluster to be
able to serve the data via NFS (or http, or anything else for that
matter really - if you can make NFS work, anything will pretty much
work) simultaneously from the same partitions, and see writes
immediately as the other hosts in the cluster commit them.
I currently have a solution just like this for Linux - Polyserve
(http://www.polyserve.com) has a clustered filesystem for linux, that
works very well. I've even tried to convince them to port it to
FreeBSD, but it falls on deaf ears, so it's time to make our own.
> Why not a port of an existing network filesystem say from Linux?
> (A BSD rewrite could be done, if the code was GPLed.) Would
> cross-platform capabilities make sense?
That would work fine I'm sure - but I have found some similar threads in
the past that claim it would be just as hard and time consuming to port
one as it would be to create one from scratch. Cross platform
capabilities would be great, but I'm mostly interested in getting
FreeBSD into this arena (as it will soon be an extremely important one
to be in).
> How do you see this comparing to device-level solutions? I know
> the argument can be made to implement file systems/storage
> abstractions at multiple layers, but I thought I might ask.
I'm not sure of a device level solution that does this. I think the OS
has to know to commit the meta-data to a journal, or otherwise let the
other machines know about locking, etc, in order for this to work.
> The other thing is there a wealth of filesystem papers out there,
> any in specific caught your eye?
No - can you point me to some?
I'll be honest here - I'm not a code developer. I would love to learn
some C here, and 'just do it', but filesystems aren't exactly simple, so
I'm looking for a group of people that would love to code up something
amazing like this - I'll support the developers and hopefully learn
something in the process. My goal personally would be to do anything I
could to make the developers work most productively, and do testing. I
can probably provide equipment, and a good testbed for it.
Eric
--
------------------------------------------------------------------------
Eric Anderson Sr. Systems Administrator Centaur Technology
A lost ounce of gold may be found, a lost moment of time never.
------------------------------------------------------------------------
More information about the freebsd-fs
mailing list