zpool - low speed write

Jeremy Chadwick freebsd at jdc.parodius.com
Thu Aug 5 05:35:06 UTC 2010


On Thu, Aug 05, 2010 at 12:47:42PM +0800, Alex V. Petrov wrote:
> > Your ada3 disk is different from the other two.  Can you please provide
> > the output from the following 3 commands?
> > 
> > camcontrol identify ada2
> > camcontrol identify ada3
> > camcontrol identify ada4
> > 
> > > vfs.zfs.cache_flush_disable=1
> > > vfs.zfs.zil_disable=1
> > 
> > I question both of these settings, especially the latter.  Please remove
> > them both and re-test your write performance.
> 
> I removed all settings of zfs.
> Now it default.
> 
> camcontrol identify ada2
> pass2: <WDC WD10EADS-00M2B0 01.00A01> ATA-8 SATA 2.x device
> pass2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
> 
> protocol              ATA/ATAPI-8 SATA 2.x
> device model          WDC WD10EADS-00M2B0
> firmware revision     01.00A01
> serial number         WD-WCAV51709425
> WWN                   50014ee2adf88aae
> cylinders             16383
> heads                 16
> sectors/track         63
> sector size           logical 512, physical 512, offset 0
> LBA supported         268435455 sectors
> LBA48 supported       1953525168 sectors
> PIO supported         PIO4
> DMA supported         WDMA2 UDMA6 
> 
> Feature                      Support  Enable    Value           Vendor
> read ahead                     yes      yes
> write cache                    yes      yes
> flush cache                    yes      yes
> overlap                        no
> Tagged Command Queuing (TCQ)   no       no
> Native Command Queuing (NCQ)   yes              32 tags
> SMART                          yes      yes
> microcode download             yes      yes
> security                       yes      no
> power management               yes      yes
> advanced power management      no       no      0/0x00
> automatic acoustic management  yes      no      254/0xFE        128/0x80
> media status notification      no       no
> power-up in Standby            yes      no
> write-read-verify              no       no      0/0x0
> unload                         no       no
> free-fall                      no       no
> data set management (TRIM)     no
> 
> *************
> 
> camcontrol identify ada3
> pass3: <WDC WD10EADS-00L5B1 01.01A01> ATA-8 SATA 2.x device
> pass3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
> 
> protocol              ATA/ATAPI-8 SATA 2.x
> device model          WDC WD10EADS-00L5B1
> firmware revision     01.01A01
> serial number         WD-WCAU4D726772
> WWN                   50014ee238ab988
> cylinders             16383
> heads                 16
> sectors/track         63
> sector size           logical 512, physical 512, offset 0
> LBA supported         268435455 sectors
> LBA48 supported       1953525168 sectors
> PIO supported         PIO4
> DMA supported         WDMA2 UDMA6 
> 
> Feature                      Support  Enable    Value           Vendor
> read ahead                     yes      yes
> write cache                    yes      yes
> flush cache                    yes      yes
> overlap                        no
> Tagged Command Queuing (TCQ)   no       no
> Native Command Queuing (NCQ)   yes              32 tags
> SMART                          yes      yes
> microcode download             yes      yes
> security                       yes      no
> power management               yes      yes
> advanced power management      no       no      0/0x00
> automatic acoustic management  yes      no      254/0xFE        128/0x80
> media status notification      no       no
> power-up in Standby            yes      no
> write-read-verify              no       no      0/0x0
> unload                         no       no
> free-fall                      no       no
> data set management (TRIM)     no
> 
> *************
> 
> camcontrol identify ada4
> pass4: <WDC WD10EADS-00M2B0 01.00A01> ATA-8 SATA 2.x device
> pass4: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
> 
> protocol              ATA/ATAPI-8 SATA 2.x
> device model          WDC WD10EADS-00M2B0
> firmware revision     01.00A01
> serial number         WD-WMAV50095864
> WWN                   50014ee014f3265
> cylinders             16383
> heads                 16
> sectors/track         63
> sector size           logical 512, physical 512, offset 0
> LBA supported         268435455 sectors
> LBA48 supported       1953525168 sectors
> PIO supported         PIO4
> DMA supported         WDMA2 UDMA6 
> 
> Feature                      Support  Enable    Value           Vendor
> read ahead                     yes      yes
> write cache                    yes      yes
> flush cache                    yes      yes
> overlap                        no
> Tagged Command Queuing (TCQ)   no       no
> Native Command Queuing (NCQ)   yes              32 tags
> SMART                          yes      yes
> microcode download             yes      yes
> security                       yes      no
> power management               yes      yes
> advanced power management      no       no      0/0x00
> automatic acoustic management  yes      no      254/0xFE        128/0x80
> media status notification      no       no
> power-up in Standby            yes      no
> write-read-verify              no       no      0/0x0
> unload                         no       no
> free-fall                      no       no
> data set management (TRIM)     no
> 
> *************
> 
> dd if=/dev/zero of=/tank/test.zero bs=3M count=1000
> 1000+0 records in
> 1000+0 records out
> 3145728000 bytes transferred in 527.727573 secs (5960894 bytes/sec)
> 
> During execution dd:
> 
> zpool iostat -v 10 10
>                capacity     operations    bandwidth
> pool         used  avail   read  write   read  write
> ----------  -----  -----  -----  -----  -----  -----
> tank        1,91T   829G      0     48  25,4K  4,35M
>   ada4       598G   333G      0     16  9,94K  1,56M
>   ada2       633G   295G      0     16  7,26K  1,54M
>   ada3       727G   201G      0     14  8,21K  1,25M
> ----------  -----  -----  -----  -----  -----  -----
> 
>                capacity     operations    bandwidth
> pool         used  avail   read  write   read  write
> ----------  -----  -----  -----  -----  -----  -----
> tank        1,91T   829G      0     29      0  2,28M
>   ada4       598G   333G      0     10      0   870K
>   ada2       633G   295G      0      8      0   774K
>   ada3       727G   201G      0      9      0   689K
> ----------  -----  -----  -----  -----  -----  -----
> 
>                capacity     operations    bandwidth
> pool         used  avail   read  write   read  write
> ----------  -----  -----  -----  -----  -----  -----
> tank        1,91T   829G      0     14      0  91,8K
>   ada4       598G   333G      0      4      0  25,2K
>   ada2       633G   295G      0      5      0  28,6K
>   ada3       727G   201G      0      4      0  38,0K
> ----------  -----  -----  -----  -----  -----  -----
> 
>                capacity     operations    bandwidth
> pool         used  avail   read  write   read  write
> ----------  -----  -----  -----  -----  -----  -----
> tank        1,91T   829G      0     19      0   227K
>   ada4       598G   333G      0      6      0  75,3K
>   ada2       633G   295G      0      6      0  76,6K
>   ada3       727G   201G      0      6      0  75,3K
> ----------  -----  -----  -----  -----  -----  -----
> 
>                capacity     operations    bandwidth
> pool         used  avail   read  write   read  write
> ----------  -----  -----  -----  -----  -----  -----
> tank        1,91T   829G      0     25      0  2,27M
>   ada4       598G   333G      0      9      0   877K
>   ada2       633G   295G      0      8      0   787K
>   ada3       727G   201G      0      7      0   659K
> ----------  -----  -----  -----  -----  -----  -----
> 
>                capacity     operations    bandwidth
> pool         used  avail   read  write   read  write
> ----------  -----  -----  -----  -----  -----  -----
> tank        1,91T   829G      0     29  12,8K  2,51M
>   ada4       598G   333G      0     11      0  1,07M
>   ada2       633G   295G      0      9      0   750K
>   ada3       727G   201G      0      8  12,8K   724K
> ----------  -----  -----  -----  -----  -----  -----
> 
>                capacity     operations    bandwidth
> pool         used  avail   read  write   read  write
> ----------  -----  -----  -----  -----  -----  -----
> tank        1,91T   829G      0     37      0  2,53M
>   ada4       598G   333G      0     12      0   847K
>   ada2       633G   295G      0     13      0   986K
>   ada3       727G   201G      0     11      0   759K
> ----------  -----  -----  -----  -----  -----  -----
> 
>                capacity     operations    bandwidth
> pool         used  avail   read  write   read  write
> ----------  -----  -----  -----  -----  -----  -----
> tank        1,91T   829G      0     25      0  2,39M
>   ada4       598G   333G      0      9      0   915K
>   ada2       633G   295G      0      8      0   900K
>   ada3       727G   201G      0      7      0   632K
> ----------  -----  -----  -----  -----  -----  -----
> 
>                capacity     operations    bandwidth
> pool         used  avail   read  write   read  write
> ----------  -----  -----  -----  -----  -----  -----
> tank        1,91T   829G      0     28      0  1,76M
>   ada4       598G   333G      0     11      0   919K
>   ada2       633G   295G      0      7      0   207K
>   ada3       727G   201G      0      9      0   676K
> ----------  -----  -----  -----  -----  -----  -----
> 
>                capacity     operations    bandwidth
> pool         used  avail   read  write   read  write
> ----------  -----  -----  -----  -----  -----  -----
> tank        1,91T   829G      0     26      0  2,16M
>   ada4       598G   333G      0     10      0   856K
>   ada2       633G   295G      0      7      0   646K
>   ada3       727G   201G      0      8      0   711K
> ----------  -----  -----  -----  -----  -----  -----

Write performance here is abysmal, agreed.  This is very odd.

I hate to say this, but can you remove ahci.ko (ahci_load="yes") from
your loader.conf and reboot?  You may need to change filesystem names
around in /etc/fstab for your OS disk (assuming it's on ada0), but for
ZFS it should just magically find the disks on adXX.

If you could also provide pciconf -lvc output that would be helpful.
Thanks.

-- 
| Jeremy Chadwick                                   jdc at parodius.com |
| Parodius Networking                       http://www.parodius.com/ |
| UNIX Systems Administrator                  Mountain View, CA, USA |
| Making life hard for others since 1977.              PGP: 4BD6C0CB |



More information about the freebsd-stable mailing list