ZFS I/O Throughput question..

[Chris Watson]

priyanka# zpool status
   pool: tank
  state: ONLINE
  scrub: none requested
config:

	NAME        STATE     READ WRITE CKSUM
	tank        ONLINE       0     0     0
	  raidz1    ONLINE       0     0     0
	    ada2    ONLINE       0     0     0
	    ada3    ONLINE       0     0     0
	    ada4    ONLINE       0     0     0
	    ada5    ONLINE       0     0     0

errors: No known data errors
priyanka#apply "dd if=/dev/ada%1 of=/dev/null bs=1m count=5000 &" 2 3  
4 5
[1] 73068
[1] 73071
[1] 73074
[1] 73077
priyanka# 5000+0 records in
5000+0 records out
5242880000 bytes transferred in 35.093718 secs (149396538 bytes/sec)
5000+0 records in
5000+0 records out
5242880000 bytes transferred in 35.374131 secs (148212262 bytes/sec)
5000+0 records in
5000+0 records out
5242880000 bytes transferred in 36.654998 secs (143033154 bytes/sec)
5000+0 records in
5000+0 records out
5242880000 bytes transferred in 38.558150 secs (135973328 bytes/sec)

priyanka# apply "dd if=/dev/zero of=/dev/ada%1 bs=1m count=5000 &" 2 3  
4 5
[1] 73130
[1] 73133
[1] 73136
[1] 73139
priyanka# 5000+0 records in
5000+0 records out
5242880000 bytes transferred in 35.149276 secs (149160398 bytes/sec)
5000+0 records in
5000+0 records out
5242880000 bytes transferred in 35.786368 secs (146504949 bytes/sec)
5000+0 records in
5000+0 records out
5242880000 bytes transferred in 36.552717 secs (143433387 bytes/sec)
5000+0 records in
5000+0 records out
5242880000 bytes transferred in 40.162832 secs (130540595 bytes/sec)
priyanka#


On Sep 17, 2010, at 3:10 PM, Pawel Jakub Dawidek wrote:

> On Fri, Sep 17, 2010 at 09:35:21PM +0200, Pawel Jakub Dawidek wrote:
>> On Wed, Sep 15, 2010 at 03:05:46AM -0500, Chris Watson wrote:
>>> I have been testing ZFS on a home box now for a few days and I  
>>> have a
>>> question that is perplexing me. Everything I have read on ZFS says  
>>> in
>>> almost every case mirroring is faster than raidz. So I initially  
>>> setup
>>> a 2x2 Raid 10 striped mirror. Like so:
>> [...]
>>
>> Could you try running something like this:
>>
>> 	# apply "dd if=/dev/ada%1 of=/dev/null bs=1m count=5000 &" 2 3 4 5
>>
>> This will tell us how much of total throughput do you have.
>> If you can destroy your data, you may also try this:
>>
>> 	# apply "dd if=/dev/null of=/dev/ada%1 bs=1m count=5000 &" 2 3 4 5
>
> 	# apply "dd if=/dev/zero of=/dev/ada%1 bs=1m count=5000 &" 2 3 4 5
>
> Thanks to se@ for noticing this.
>
>> If you disks cannot work at full speed in parallel this might explain
>> what you're seeing. Mirror send to disk twice as much data as it
>> receives and RAIDZ sends only 33% more data in four disk case.
>>
>> And no, there are neither special RAIDZ optimizations not special  
>> mirror
>> pesimizations in FreeBSD.
>
> -- 
> Pawel Jakub Dawidek                       http://www.wheelsystems.com
> pjd at FreeBSD.org                           http://www.FreeBSD.org
> FreeBSD committer                         Am I Evil? Yes, I Am!