UFS Filesystem issues, and the loss of my hair...
Matt Simerson
matt at corp.spry.com
Fri Aug 7 20:46:07 UTC 2009
On Aug 6, 2009, at 6:51 AM, Hearn, Trevor wrote:
> First off, let me state that I love FreeBSD. I've used it for years,
> and have not had any major problems with it... Until now.
>
> As you can tell, I work for a major university. I setup a large
> storage array to hold data for a project they have here. No great
> shakes, just some standard files and such.
> <snip>
> I'd buy a fella, or gal, a cup of coffee and a pop-tart if they
> could help a brother out. I have checked out this link:
> http://phaq.phunsites.net/2007/07/01/ufs_dirbad-panic-with-mangled-entries-in-ufs/
> and decided that I need to give this a shot after hours, but being
> the kinda guy I am, I need to make sure I am covering all of my bases.
>
> Anyone got any ideas?
>
> Thanks!
Have you given any consideration to ZFS?
With ZFS there's no reason to have all those slices. Just stripe the
two RAID 6 arrays together and have a single 26TB zpool. No GPT or UFS
to mess with. Just point ZFS at the raw disks and off you go. I'm
doing that with Areca 1231ML controllers in boxes with 24 disks each.
The two 12 channel RAID cards each present a RAID volume to the OS and
zpool stripes them together.
One of the more useful features of ZFS is file system compression. You
may find that with file system compression, you can get by with 13TB
of storage. Then you have one RAID 6 array as the data store and the
2nd array for backups on each machine. With ZFS, you can send
snapshots of the data partition to the backup every hour, or even
every minute without any appreciable impact.
back01# zfs get compression back01/var
NAME PROPERTY VALUE SOURCE
back01/var compression gzip local
back01# zfs get compressratio back01/var
NAME PROPERTY VALUE SOURCE
back01/var compressratio 2.16x -
I'm using gzip compression and I fit over twice as much data on the
filesystem as I'd otherwise be getting. You can get more aggressive
with gzip-9 if you need.
You could use your backup server as a proof-of-concept. Install
FreeBSD 8-BETA2 amd64 on it. Unmount the existing GPT partitions, wipe
the MBR clean using dd, and create a zpool on just one of the RAID 6
volumes. Set ZFS compression=gzip on your filesystem and use rsync to
copy all the files from your 'primary' server. I suspect you'll find
that you have ample storage. Then you can create another zpool on that
same box using the other RAID 6 volume for backups. You can experiment
there with zfs send/receive, or rsnapshot, or whatever you use.
Then get a subset of your users to start testing on it and see how it
fares. I suspect you'll be quite pleased. If it works out wonderfully,
you can rebuild the other GPT/UFS system on ZFS as well. Set it up
with both RAID 6 volumes in one ZFS pool and start pushing your
backups from the primary server to it. Once successfully backed up,
you can add the 2nd RAID 6 volume on the primary server into the
storage pool to double it's disk space.
Matt
More information about the freebsd-fs
mailing list