Poor I/O performance; maybe ZFS?

[Julio Merino]

Hello,

I have a PowerMac G5 running 11.0-CURRENT (upgraded yesterday).  The machine is a dual-CPU 2GHz G5 with 6GB of RAM and two drives:

ada0 at ata2 bus 0 scbus2 target 0 lun 0cd0 at ata0 bus 0 scbus0 target 0 lun 0
ada0: <ST3160023AS 3.05> ATA-6 SATA 1.x device
ada0: Serial Number 3JS478VL
ada0: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes)
ada0: 152627MB (312581808 512 byte sectors: 16H 63S/T 16383C)
ada1 at ata3 bus 0 scbus3 target 0 lun 0
ada1: <WDC WD6400AAKS-75A7B0 01.03B01> ATA8-ACS SATA 2.x device
ada1: Serial Number WD-WMASY2696897
ada1: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes)
ada1: 610480MB (1250263728 512 byte sectors: 16H 63S/T 16383C)

I have a UFS file system on ada0s4 for / and a ZFS pool on ada1s2 with file systems for /home and poudriere.

For a while, I have felt that the system is incredibly sluggish.  For example, a simple "make buildworld" with up-to-date obj, which should be I/O bound, takes forever (something like 20-30 minutes).  This is running with src and obj on ZFS.  "svn update" is similarly infuriatingly slow.

Today, I decided to install bonnie and ran a couple of tests using "-s 4096".

              -------Sequential Output-------- ---Sequential Input-- --Random--
              -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
ada0     4196 43973 62.4 46207 37.7 43259 41.9 89210 96.8 280589 95.9 17038.2 103.1
tank     4196 54093 72.2 27596 93.0 21478 90.7 64214 83.5 65358 50.8 126.7  3.7

Note the vast difference on the numbers, especially random seeks.

In case this matters, this is how ada1s2 looks like:

1. Name: ada1s2
   Mediasize: 640135019520 (596G)
   Sectorsize: 512
   Stripesize: 0
   Stripeoffset: 9216
   Mode: r1w1e1
   rawtype: FreeBSD-ZFS
   label: 1
   length: 640135019520
   offset: 9216
   type: freebsd-zfs
   index: 1
   end: 1250263727
   start: 18

Any obvious things I should look at?

Thank you!