Filesystem, RAID Question

[Matthew Seaman]

Rich Fairbanks wrote:

> Now, this is how I set up the array. I installed the card, popped in the
> drives. The card bios found the drives and allowed me to setup in RAID 5.
> Then, FreeBSD booted and found the "disk" as da0. I want the entire array to
> be one big chunk of space. In other words, I don't need a bunch of slices or
> partitions (or DO I? I'm still very new to the whole slice vs. partition
> concept)

The default settings should actually work just about right for a 
general purpose file system with reasonably sized files.  A RAID5 
across 3x1TB drives will give you 2+ TB usable space -- that's within 
the  capabilities of UFS2, so you should be OK there.  However a 3 disk 
RAID5 is the worst performing RAID5 setup you can create.  A larger 
number of smaller disks would probably have served you better.

> I typed newfs /dev/da0 . A ton of numbers went across the screen, then I
> mounted /dev/da0 at /usr/home/storage. It works, but perhaps I missed a step
> that would have made things easier/perform better, etc.

The sort of changes you can make at newfs time mostly affect how 
efficient the storage is -- ie. tuning the system for particularly 
large or small files.  While newfs and tunefs can affect performance, 
they aren't the first thing to look at here. 

> Besides creating the file system a different way, what would be an optimum
> stripe size for the array? I will using this for storing, basically, a TON
> of word documents and email messages, and a few large .pst files. So, the
> average file size will be in the 25-100K range, but a few 1-2GB files.

Just take the default stripe size the array controller presents you 
with -- it will be appropriate for this sort of mixed file sizes.

The first thing to consider is what sort of IO caching strategy your
hardware is using.  Does your RAID controller have a battery backup
unit?  Probably not, as that tends to add a large whack onto the price.

If not, then your array controller will not report an IO operation as 
complete to the OS until the bits have been written to the disk[*].  
With the BBU, the controller can report the operation as complete as 
soon as  the data is stored in (battery backed) RAM on the controller.  
These  modes are called 'write through' and 'write back' in some 
controllers, but I can't for the life of me remember which is which.

Given that you don't have a BBU, what is the status of write caching
on the individual hard drives?  You'll have to use 3dm2 or the CLI 
equivalent to investigate this, as the RAID controller tends to hide 
that level of information from the OS.  However, this setting is the
same thing as controlled by the hw.ata.wc sysctl -- and like that 
it has a major effect on disk IO performance.  Turning write caching 
off is the safe, conservative thing to do for maximum data security.  

Turning write caching on is the only way to get decent performance out 
of ordinary hard drives, but it leaves you open to data loss if the 
machine should crash or lose power suddenly.  Most systems with ATA
or ordinary SATA drives default to using write caching.  SCSI and fast
SAS drives can be configured either way.

You'ld always turn disk level write caching off if you've got a BBU, 
because it's made redundant in that case by the controller memory 
cache.

If fiddling with write caching can't make things any better, then I'd 
reconsider using RAID5.  Unfortunately 3 disks doesn't leave you with 
many options.  Add another drive of the same size and you can make a 4 
disk RAID10 with 2TB usable space.  Or you can configure the RAID 
controller to act as a JBOD and try out ZFS -- the RAID-Z mode is 
the moral equivalent of RAID5 but quite different in operation.

	Cheers,

	Matthew

[*] Some disks have been known to lie about completing IO transactions 
even when set to the most conservative mode.  IMHO they aren't fit for 
purpose and should you be landed with such things you'ld be entitled 
to a refund from the vendor.

-- 
Dr Matthew J Seaman MA, D.Phil.                   7 Priory Courtyard
                                                  Flat 3
PGP: http://www.infracaninophile.co.uk/pgpkey     Ramsgate
                                                  Kent, CT11 9PW

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 258 bytes
Desc: OpenPGP digital signature
Url : http://lists.freebsd.org/pipermail/freebsd-questions/attachments/20081029/46be7a9e/signature.pgp