Performance problems on LSI cards.. SOLVED.

[Michelle Sullivan]

A long time ago I posted about my ZFS woes (which I might add are fixed 
thanks to the 'zfs-recovery' tool)

...but throughout since 2011 I have had performance issues with my 
LSI-9260-16i cards now I finally have solved the performance problems 
resulting in maximum scrub speed of 67M/s...  (its a 15disk raidz2 setup 
with Seagate SATAIII x 3T drives)...

2 problems, one hid the other... first turn off bios for presenting the 
drives to the OS ... exactly why this had an impact I'm not sure but I 
think I can guess based on what actually fixed it...

My systems have 6 PCIe slots. 3 off PCIe, 1 off PCIe2x16 and 2 off 
PCIe3/2x16 (or 2x8) ... specifically the manual says:


2 x PCI Express 3.0/2.0 x16 slots (single at x16 or dual at x8/x8 mode)

1 x PCI Express 2.0 x16 slot* (max. at x4 mode, compatible with PCIe x1 
and x4 devices)

3 x PCI Express 2.0 x1 slots**

* The PCIe 2.0 x16 slot shares bandwidth with PCIe 2.0 x1_1 slot, PCIe 
2.0 x1_2 slot, and PCIe 2.0 x1_3 slot. The PCIe 2.0 x16 slot default 
setting is x1 mode.

** The PCIe 2.0 x1_1 slot shares bandwidth with eSATA 6.0 Gb/s ports. 
The eSATA 6.0 Gb/s ports default settings are enabled.

Being there is a video card in appropriate PCIe3 slot and the card is 
PCIe2 I put them in non shared slots...  If anyone else is looking at 
doing similar... don't .. the cards are 8 channel and they (not sure why 
- probably the firmware - forcing to x4 didn't work either - all other 
slots are empty so no sharing for that slot) seem to work at 8 channel 
or 1 channel.  Putting the video card and the LSI in the PCI3/2 x16 
slots and switching x8/x8 mode and the scrubs are now up at 421M/s 
(average.)

Anyhow I'll go bck to my corner now.

Regards,

-- 
Michelle Sullivan
http://www.mhix.org/