More Controllers != Higher Through Put

[Stephen Sanders]

I'm wondering if anyone has heard of this.

I've a system with a 3ware 9650 servicing 4 7200RPM Segate 1TB drives
and the motherboard servicing 2 7200 RPM Segate 1TB drives.

The 4 disk array is RAID 6 while the 2 disk array is RAID 1.  The drives
should deliver about 100MB/s.

1.  The most the 4 disk array is developing is 250MB/s write performance
while the 2 disk array is coming in at 90MB/s write performance.  The 4
disk array seems slow.

2. Attempting to write to both arrays simultaneously causes the rate on
the 4 disk array to drop to 150MB/s and the 2 disk array drops to 60MB/s

I'd expect the 4 disk array should look more like 300+MB/s while the 2
disk array is about right.

I don't get why there should be a 'coupling' between the rates on
separate controllers. 

The system is running FreeBSD 8.0, has 16GB of RAM in the system, and
the test program is using O_DIRECT for writes in order to avoid the page
daemon.

Thanks