FS of choice for max random iops ( Maildir )

[freebsd at top-consulting.net]

Quoting Terje Elde <terje at elde.net>:

> On 16. sep. 2011, at 12:31, freebsd at top-consulting.net wrote:
>> Right now I defined an entire array of 8TB ( all 16 disks )  
>> separated in two pieces. 50 GB for FreeBSD to boot and the rest  
>> available to configure as storage.
>
> ZFS will want to write to it's ZIL (zfs intent log) before writing  
> to the final location of the data. Even if you're not waiting for  
> the ZIL-write to disk (because of the controller ram), those writes  
> will probably make it through to disk. That gives you twice as many  
> writes to disk, and a lot more seek.
>
> If you want to take zfs for a proper spin, I'd like to sugget adding  
> two small SSDs to the setup, mirrored by zfs. You can use those both  
> for the ZIL, and also as cache, for the array. That's a fairly small  
> investment these days, and I would be surprised if it didn't  
> significantly improve performance, both for your benchmark, and real  
> load.
>
> Note: you might be in trouble if you loose your ZIL, thus the  
> doubling up. I *think* you can SSD a cache without risking dataloss,  
> but don't take my word for it.
>
> Terje

Well, I tried disabling the ZIL on a new dataset. These are the  
commands that I ran:

zpool create data da1
zfs create data/maildomains
zfs set sync=disabled data/maildomains

dd bs=1024 if=/dev/zero of=/data/maildomains/t1 count=1M
1048576+0 records in
1048576+0 records out
1073741824 bytes transferred in 14.537711 secs (73859071 bytes/sec)

Got a measly 74MB/sec.

On the UFS partition however...

dd bs=1024 if=/dev/zero of=/usr/t1 count=1M
1048576+0 records in
1048576+0 records out
1073741824 bytes transferred in 5.828395 secs (184225983 bytes/sec)

184MB/sec!

And this is synchronous writing, not random!

So what is ZFS good for finally ? Synchronous writing or small random iops ?

By the way, this is how the array is configured with 3ware:

Unit  UnitType  Status         %RCmpl  %V/I/M  Stripe  Size(GB)  Cache  AVrfy
------------------------------------------------------------------------------
u0    RAID-10   OK             -       -       64K     7450.5    ON     ON

VPort Status         Unit Size      Type  Phy Encl-Slot    Model
------------------------------------------------------------------------------
p0    OK             u0   931.51 GB SATA  0   -            WDC WD1002FBYS-01A6
p1    OK             u0   931.51 GB SATA  1   -            WDC WD1002FBYS-01A6
p2    OK             u0   931.51 GB SATA  2   -            WDC WD1002FBYS-01A6
p3    OK             u0   931.51 GB SATA  3   -            WDC WD1002FBYS-01A6
p4    OK             u0   931.51 GB SATA  4   -            WDC WD1002FBYS-01A6
p5    OK             u0   931.51 GB SATA  5   -            WDC WD1002FBYS-01A6
p6    OK             u0   931.51 GB SATA  6   -            WDC WD1002FBYS-01A6
p7    OK             u0   931.51 GB SATA  7   -            WDC WD1002FBYS-01A6
p8    OK             u0   931.51 GB SATA  8   -            WDC WD1002FBYS-01A6
p9    OK             u0   931.51 GB SATA  9   -            WDC WD1002FBYS-01A6
p10   OK             u0   931.51 GB SATA  10  -            WDC WD1002FBYS-01A6
p11   OK             u0   931.51 GB SATA  11  -            WDC WD1002FBYS-01A6
p12   OK             u0   931.51 GB SATA  12  -            WDC WD1002FBYS-01A6
p13   OK             u0   931.51 GB SATA  13  -            WDC WD1002FBYS-01A6
p14   OK             u0   931.51 GB SATA  14  -            WDC WD1002FBYS-01A6
p15   OK             u0   931.51 GB SATA  15  -            WDC WD1002FBYS-01A6