large RAID volume partition strategy

[Boris Samorodov]

On Sat, 18 Aug 2007 02:26:04 +0400 Boris Samorodov wrote:
> On Fri, 17 Aug 2007 17:42:55 -0400 Vivek Khera wrote:

> > I have a shiny new big RAID array.  16x500GB SATA 300+NCQ drives
> > connected to the host via 4Gb fibre channel.  This gives me 6.5Tb of
> > raw disk.

> > I've come up with three possibilities on organizing this disk.  My
> > needs are really for a single 1Tb file system on which I will run
> > postgres.  However, in the future I'm not sure what I'll really need.
> > I don't plan to ever connect any other servers to this RAID unit.

> > The three choices I've come with so far are:

> > 1) Make one RAID volume of 6.5Tb (in a RAID6 + hot spare
> > configuration), and make one FreeBSD file system on the whole
> > partition.

> > 2) Make one RAID volume of 6.5Tb (in a RAID6 + hot spare
> > configuration), and make 6 FreeBSD partitions with one file system
> > each.

> > 3) Make 6 RAID volumes and expose them to FreeBSD as multiple drives,
> > then make one partition + file system on each "disk".  Each RAID
> > volume would span across all 16 drives, and I could make the volumes
> > of differing RAID levels, if needed, but I'd probably stick with RAID6
> > +spare.

> > I'm not keen on option 1 because of the potentially long fsck times
> > after a crash.

> > What advantage/disadvantage would I have between 2 and 3?  The only
> > thing I can come up with is that the disk scheduling algorithm in
> > FreeBSD might not be optimal if the drives really are not truly
> > independent as they are really backed by the same 16 drives, so
> > option 2 might be better.  However, with option 3, if I do ever end
> > up connecting another host to the array, I can assign some of the
> > volumes to the other host(s).

> > My goal is speed, speed, speed.

> Seems that RAID[56] may be too sloooow. I'd suggest RAID10.

> I have 6 SATA-II 300MB/s disks at 3WARE adapter. My (very!) simple
> tests gave about 170MB/s for dd. BTW, I tested (OK, very fast)
> RAID5, RAID6, gmirror+gstripe and noone get close to RAID10. (Well, as
> expected, I suppose).

> > I'm running FreeBSD 6.2/amd64 and
> > using an LSI fibre card.

> If you have time you may do your own tests... And in case RAID0 you
                                                            ^^^^^
RAID10

> shouldn't have problems with long fsck. Leave a couple of your disks
> for hot-swapping and you'll get 7Tb. ;-)
                                  ^^^
3.5TB

> > Thanks for any opinions and recommendations.

sorry, not my night...


WBR
-- 
bsam