filesystem: 12h to delete 32GB of data
fjwcash at gmail.com
Wed May 6 20:51:10 UTC 2009
On Wed, May 6, 2009 at 12:21 PM, Matthew Seaman
<m.seaman at infracaninophile.co.uk> wrote:
> Gary Gatten wrote:
>> OT now, but in high i/o envs with high concurrency needs, RAID5 is
>> still the way to go, esp if 90% of i/o is reads. Of course it depends
>> on file size / type as well... Anyway, let's sum it up with "a
>> storage subsystem is only as fast as its slowest link"
> It's not just the balance of reads over writes. It's the size and
> sequential location of the IO requests. RAID5 is good for sequential reads -- eg.
> streaming a video -- where the system can read whole blocks from all the
> drives involved, calculate parity over the whole lot and then push all that
> blob of data up to the CPU.
> RAID5 is pretty pessimal if your usage pattern is small reads or writes
> randomly scattered over your storage area -- eg. typical RDBMS behaviour
> -- which works a great deal better on RAID10.
> I'd also contend that the essential difference between a really good fast
> hardware raid controller and something disappointingly mundane is a decent
> amount of non-volatile cache memory. For most H/W raid that equates to
> using a battery backup unit. I've been thinking though that a few GB of
> fast solid-state hard drive configured as a gjournal for a RAID10 (ie
> gstripe +gmirror) might achieve the same effect for rather less outlay... It
> would probably not be too shabby with RAID5 even, but of course you'ld
> lose the benefit of offloading parity calculations onto the RAID
> controller's CPU. Still, modern multi-core CPUs are probably fast enough nowadays to
> make that viable for many purposes.
Depending on the number of drives you are using, ZFS would also be
worth looking at. The raidz implementation works quite nicely, and
(in theory) doesn't suffer from the major issues that RAID5/6 does.
It also does implicit striping across all vdevs, so you can make some
very fancy RAID layouts (each vdev can be mirrored, raidz1, raidz2, or
just a bunch of disks).
I don't know if the version of ZFS in FreeBSD 7.x supports hybrid
pools, but the version in FreeBSD 8.0 should, which lets you add SSDs
to the pool to be used automatically as "cache" in-between RAM and
fjwcash at gmail.com
More information about the freebsd-performance