aac scsi raid driver performance

[Ahnjoan Amous]

I'm trying to find possible explanations for slow concurrent writes through the
aac driver.  This machine runs under 1% load and has less than 4 transfers per
second to the drives in question when not being used for testing.

When I attempt sequential "dd"s as follow, the results are better then 70MB/sec.
  dd if=/dev/zero of=/data02/helloworld bs=1m count=1000
    1048576000 bytes transferred in 13.886718 secs (75509274 bytes/sec)
  dd if=/dev/zero of=/data03/helloworld bs=1m count=1000
    1048576000 bytes transferred in 14.011323 secs (74837758 bytes/sec)

When I attempt concurrent "dd"s as follow, with a 1 second sleep interval
between starts, the results are better than 40MB/sec
  dd if=/dev/zero of=/data02/helloworld bs=1m count=1000 &
  sleep 1
  dd if=/dev/zero of=/data03/helloworld bs=1m count=1000 &
    1048576000 bytes transferred in 25.269555 secs (41495626 bytes/sec)
    1048576000 bytes transferred in 24.935765 secs (42051086 bytes/sec)

When I attempt concurrent "dd"s as follow, the results are little better than
20MB/sec
  dd if=/dev/zero of=/data02/helloworld bs=1m count=1000 &
  dd if=/dev/zero of=/data03/helloworld bs=1m count=1000 &
    1048576000 bytes transferred in 44.963408 secs (23320652 bytes/sec)
    1048576000 bytes transferred in 45.010065 secs (23296478 bytes/sec)

I can't account for what causes the huge difference however the results are
reproducible.  I've run the tests dozens and dozens of times now, first blaming
the em driver for my slow ggatec/ggated results, then GEOM for my slow local
mirroring after eliminating the network, and finally blaming the aac driver
after removing GEOM from the equation.  If anyone has ideas on what I might
look at or change or test I would love to hear.


****** Misc. Information ******
root:somehost:~ > df -k
Filesystem    1K-blocks    Used  Avail Capacity  Mounted on
/dev/aacd2s1e   2026030 1024532 839416    55%    /data02
/dev/aacd3s1e   2026030 1024532 839416    55%    /data03
Hardware -
aac - Dell PERC3/Di U160
aacd2 - hardware RAID 0, w/1 U320 300G drive
aacd3 - hardware RAID 0, w/1 U320 300G drive