8.0-RELEASE/amd64 - full ZFS install - low read and write disk performance

[Dan Naumov]

On Mon, Jan 25, 2010 at 2:14 AM, Dan Naumov <dan.naumov at gmail.com> wrote:
> On Sun, Jan 24, 2010 at 11:53 PM, Alexander Motin <mav at freebsd.org> wrote:
>> Dan Naumov wrote:
>>> This works out to 1GB in 36,2 seconds / 28,2mb/s in the first test and
>>> 4GB in 143.8 seconds / 28,4mb/s and somewhat consistent with the
>>> bonnie results. It also sadly seems to confirm the very slow speed :(
>>> The disks are attached to a 4-port Sil3124 controller and again, my
>>> Windows benchmarks showing 65mb/s+ were done on exact same machine,
>>> with same disks attached to the same controller. Only difference was
>>> that in Windows the disks weren't in a mirror configuration but were
>>> tested individually. I do understand that a mirror setup offers
>>> roughly the same write speed as individual disk, while the read speed
>>> usually varies from "equal to individual disk speed" to "nearly the
>>> throughput of both disks combined" depending on the implementation,
>>> but there is no obvious reason I am seeing why my setup offers both
>>> read and write speeds roughly 1/3 to 1/2 of what the individual disks
>>> are capable of. Dmesg shows:
>>>
>>> atapci0: <SiI 3124 SATA300 controller> port 0x1000-0x100f mem
>>> 0x90108000-0x9010807f,0x90100000-0x90107fff irq 21 at device 0.0 on
>>> pci4
>>> ad8: 1907729MB <WDC WD20EADS-32R6B0 01.00A01> at ata4-master SATA300
>>> ad10: 1907729MB <WDC WD20EADS-00R6B0 01.00A01> at ata5-master SATA300
>>
>> 8.0-RELEASE, and especially 8-STABLE provide alternative, much more
>> functional driver for this controller, named siis(4). If your SiI3124
>> card installed into proper bus (PCI-X or PCIe x4/x8), it can be really
>> fast (up to 1GB/s was measured).
>>
>> --
>> Alexander Motin
>
> Sadly, it seems that utilizing the new siis driver doesn't do much good:
>
> Before utilizing siis:
>
> iozone -s 4096M -r 512 -i0 -i1
>                                                            random
> random    bkwd   record   stride
>              KB  reclen   write rewrite    read    reread    read
> write    read  rewrite     read   fwrite frewrite   fread  freread
>         4194304     512   28796   28766    51610    50695
>
> After enabling siis in loader.conf (and ensuring the disks show up as ada):
>
> iozone -s 4096M -r 512 -i0 -i1
>
>                                                            random
> random    bkwd   record   stride
>              KB  reclen   write rewrite    read    reread    read
> write    read  rewrite     read   fwrite frewrite   fread  freread
>         4194304     512   28781   28897    47214    50540

Just to add to the numbers above, exact same benchmark, on 1 disk
(detached 2nd disk from the mirror) while using the siis driver:

                                                            random
random    bkwd   record   stride
              KB  reclen   write rewrite    read    reread    read
write    read  rewrite     read   fwrite frewrite   fread  freread
         4194304     512   57760   56371    68867    74047


- Dan