Multi node storage, ZFS

Michal michal at
Wed Mar 24 15:47:29 UTC 2010

I wrote a really long e-mail but realised I could ask this question far
far easier, if it doesn't make sense, the original e-mail is bellow

Can I use ZFS to create a multinode storage area. Multiple HDD's in
Multiple servers to create one target of, for example, //officestorage
Allowing me to expand the storage space when needed and clients being
able to retrieve data (like RAID0 but over devices not HDD)

Here is an example I found which is where I'm getting some ideas from

Any pointers would be helpful,


I've been searching around and I am finding myself confused and reading
conflicting information. I would like to build a Storage system where by
I have multiple nodes. At the minute I have a number or NAS's which work
well and RAID6 works well in the situation we have, but unfortunately
it's a short-term solution I inherited and once you crunch the numbers
of 6 devices with 6 HDD's in RAID6 you realise how much space you have
wasted then say, 1 device of RAID6 with 36 HDD's (the saving is a fair
few TB)

There are other issues as well, increasing the size, 3rd party NAS
device features missing which other storage devices have...etc so I
looked around and my grand idea was basically this;

"Build a system where I can have multiple nodes which create one target
(we will use //officestorage for our example) as opposed to //nas1/
which is of course 1 device. Using multiple nodes will allow us to add a
new device, thus increasing the space available but the target will
always be the same and to the client nothing has change (other then
available space) (Think of this as RAID0). Multiple nodes will also
allow for redundancy across devices (think RAID1) and give better IO as
it's multiple devices and not just 1 device. I could have devices in
different locations so a whole building could burn down and still not
lose the data"

After looking around I found this

Which looks quite good, it's basically RAID1 but instead of HDD's it's
across servers, I've used DRBD and it worked well, but this doesn't give
me better IO as only 1 device is live.

I then found this

Which looks fantastic, though I will need a master server to create the
RAID0 and RAID1 across the multiple nodes and then share this out, which
is ok but I would need a hot swap master server, so I'm looking at two
of those. I then started thinking about ZFS, I've heard lots of good
things about it in the past and thinking can ZFS do what I want. I have
read some things which say it can do what that 2nd link does and others
which say it can't. Everything I come across is about using just 1
device and I could build 1 device with DRBD, but that doesn't help, nor
will it allow me to expand it (if your server runs out of physical space
you can't add more HDD's.

Anyone point me in the right direction??


More information about the freebsd-stable mailing list