8.0-RELEASE/amd64 - full ZFS install - low read and write disk performance

[Alexander Motin]

Dan Naumov wrote:
> On Mon, Jan 25, 2010 at 2:14 AM, Dan Naumov <dan.naumov at gmail.com> wrote:
>> On Sun, Jan 24, 2010 at 11:53 PM, Alexander Motin <mav at freebsd.org> wrote:
>>> Dan Naumov wrote:
>>>> This works out to 1GB in 36,2 seconds / 28,2mb/s in the first test and
>>>> 4GB in 143.8 seconds / 28,4mb/s and somewhat consistent with the
>>>> bonnie results. It also sadly seems to confirm the very slow speed :(
>>>> The disks are attached to a 4-port Sil3124 controller and again, my
>>>> Windows benchmarks showing 65mb/s+ were done on exact same machine,
>>>> with same disks attached to the same controller. Only difference was
>>>> that in Windows the disks weren't in a mirror configuration but were
>>>> tested individually. I do understand that a mirror setup offers
>>>> roughly the same write speed as individual disk, while the read speed
>>>> usually varies from "equal to individual disk speed" to "nearly the
>>>> throughput of both disks combined" depending on the implementation,
>>>> but there is no obvious reason I am seeing why my setup offers both
>>>> read and write speeds roughly 1/3 to 1/2 of what the individual disks
>>>> are capable of. Dmesg shows:
>>>>
>>>> atapci0: <SiI 3124 SATA300 controller> port 0x1000-0x100f mem
>>>> 0x90108000-0x9010807f,0x90100000-0x90107fff irq 21 at device 0.0 on
>>>> pci4
>>>> ad8: 1907729MB <WDC WD20EADS-32R6B0 01.00A01> at ata4-master SATA300
>>>> ad10: 1907729MB <WDC WD20EADS-00R6B0 01.00A01> at ata5-master SATA300
>>> 8.0-RELEASE, and especially 8-STABLE provide alternative, much more
>>> functional driver for this controller, named siis(4). If your SiI3124
>>> card installed into proper bus (PCI-X or PCIe x4/x8), it can be really
>>> fast (up to 1GB/s was measured).
>>>
>>> --
>>> Alexander Motin
>> Sadly, it seems that utilizing the new siis driver doesn't do much good:
>>
>> Before utilizing siis:
>>
>> iozone -s 4096M -r 512 -i0 -i1
>>                                                            random
>> random    bkwd   record   stride
>>              KB  reclen   write rewrite    read    reread    read
>> write    read  rewrite     read   fwrite frewrite   fread  freread
>>         4194304     512   28796   28766    51610    50695
>>
>> After enabling siis in loader.conf (and ensuring the disks show up as ada):
>>
>> iozone -s 4096M -r 512 -i0 -i1
>>
>>                                                            random
>> random    bkwd   record   stride
>>              KB  reclen   write rewrite    read    reread    read
>> write    read  rewrite     read   fwrite frewrite   fread  freread
>>         4194304     512   28781   28897    47214    50540
> 
> Just to add to the numbers above, exact same benchmark, on 1 disk
> (detached 2nd disk from the mirror) while using the siis driver:
> 
>                                                             random
> random    bkwd   record   stride
>               KB  reclen   write rewrite    read    reread    read
> write    read  rewrite     read   fwrite frewrite   fread  freread
>          4194304     512   57760   56371    68867    74047

If both parts of mirror uses same controller, it doubles it's bus
traffic. That may reduce bandwidth twice.

The main benefit of siis(4) is a command queuing. You should receive
more benefits on multithread random I/O.

-- 
Alexander Motin