24 TB UFS2 reality check ?

[Gary Palmer]

On Thu, Jul 10, 2008 at 08:55:17AM -0700, Juri Mianovich wrote:
> 
> Jeff,
> 
> 
> --- On Wed, 7/9/08, Jeff Mohler <speedtoys.racing at gmail.com> wrote:
> 
> > Lets see..a peak of maybe 25-30 random drive IOPS/sec at
> > 15ms MINIMAL
> > latency per IO (likely more like 35-40)..gonna be ugly.
> > 
> > Complicated by normal load IOPS..you could expect it all to
> > simply
> > "dissapear" for a day while it reconstructs.
> 
> 
> Once again, thank you very much - your comments are very helpful.
> 
> So we've moved from "dangerous" (24 TB with no raid) to "inconvenient" (24 TB with raid 6).
> 
> Two final questions:
> 
> 1. What would _you_ do with 24 1 TB disks and a 24 port 3ware card ? 
> Assume an i386, 4 GB machine, and that fsck is workable because of 
> "newfs -i 131072"
> 
> 2. What number should I ask my vendor (3ware) to do the rebuild
> calculations ?  You are talking about IOPS/s - I think I should ask
> them how many IOPS/s the card does when rebuilding a 24 disk raid-6
> array, and then combine that with the IOPS/s I see in my normal workload.
>

At least on older 3Ware cards, and I suspect the one you're talking
about will do it also, you can control the rebuild rate.  There are
5 different options provided on my card (8506-4LP) for "background task
rate" which either increases or decreases the IOPS used for
the rebuild.  

> How do you measure IOPS/s in FreeBSD on a running machine ?

iostat and/or systat are the two I use.

Regards,

Gary