LSI supported mps(4) driver available

Thu Jan 26 16:29:28 UTC 2012

Kenneth D. Merry schreef:
> On Tue, Jan 24, 2012 at 15:42:57 +0100, Johan Hendriks wrote:
>> Kenneth D. Merry schreef:
>>> The LSI-supported version of the mps(4) driver that supports their 6Gb SAS
>>> HBAs as well as WarpDrive controllers, is available here:
>>>
>>> http://people.freebsd.org/~ken/lsi/mps_lsi.20120120.1.txt
>>>
>>> I plan to check it in to head next week, and then MFC it into stable/9 a
>>> week after that most likely.
>>>
>>> Please test it out and let me know if you run into any problems.
>>>
>>> In addition to supporting WarpDrive, the driver also supports Integrated
>>> RAID.
>>>
>>> Thanks to LSI for doing the work on this driver!
>>>
>>> I have added a number of other infrastructure changes that are necessary
>>> for the driver, and here is a brief summary:
>>>
>>>   - A new Advanced Information buffer is now added to the EDT for drives
>>>     that support READ CAPACITY (16).  The da(4) driver updates this buffer
>>>     when it grabs new read capacity data from a drive.
>>>   - The mps(4) driver will look for Advanced Information state change async
>>>     events, and updates its table of drives with protection information
>>>     turned on accordingly.
>>>   - The size of struct scsi_read_capacity_data_long has been bumped up to
>>>     the amount specified in the latest SBC-3 draft.  The hope is to avoid
>>>     some future structure size bumps with that change.  The API for
>>>     scsi_read_capacity_16() has been changed to add a length argument.
>>>     Hopefully this will future-proof it somewhat.
>>>   - __FreeBSD_version bumped for the addition of the Advanced Information
>>>     buffer with the read capacity information.  The mps(4) driver has a
>>>     kludgy way of getting the information on versions of FreeBSD without
>>>     this change.
>>>
>>> I believe that the CAM API changes are mild enough and beneficial enough
>>> for a merge into stable/9, but they are intertwined with the unmap changes
>>> in the da(4) driver, so those changes will have to go back to stable/9 as
>>> well in order to MFC the full set of changes.
>>>
>>> Otherwise it'll just be the driver that gets merged into stable/9, and
>>> it'll use the kludgy method of getting the read capacity data for each
>>> drive.
>>>
>>> A couple of notes about issues with this driver:
>>>
>>>   - Unlike the current mps(4) driver, it probes sequentially.  If you have
>>>   a
>>>     lot of drives in your system, it will take a while to probe them all.
>>>   - You may see warning messages like this:
>>>
>>> _mapping_add_new_device: failed to add the device with handle 0x0019 to
>>> persiste
>>> nt table because there is no free space available
>>> _mapping_add_new_device: failed to add the device with handle 0x001a to
>>> persiste
>>> nt table because there is no free space available
>>>
>>>   - The driver is not endian safe.  (It assumes a little endian machine.)
>>>     This is not new, the driver in the tree has the same issue.
>>>
>>> The LSI folks know about these issues.  The driver has passed their testing
>>> process.
>>>
>>> Many thanks to LSI for going through the effort to support FreeBSD.
>>>
>>> Ken
>> Sorry to bother you with this, but how do i test this on a 9.0 release
>> machine.
>> I am no developer what so ever, but we use some LSI 9211-8i cards, in
>> supermicro storage machines.
>> One machine give me scsi timeouts, and i like to try the new driver.
>>
>> I did the following: save the
>> http://people.freebsd.org/~ken/lsi/mps_lsi.20120120.1.txt as lsi.patch
>> in /root
>>
>> then
>> # cd /usr
>> # patch<  lsi.patch
>> I do get some .rej from patch
>>
>> But when i rebuild the kernel it errors out, so i think i am doing
>> something wrong.
> The overall patch won't apply cleanly to 9.0, because it depends on some
> other CAM changes that are not in 9.0.
>
> The driver itself may apply and work, however.
>
> The easiest thing to do may be to edit the patch and only keep the patches
> against files in sys/dev/lsi, sys/conf, and sys/modules.
>
> Before you apply the revised patch, make sure you either back out the
> previous patch attempt, or otherwise get a clean source tree.
>
> Ken
Ok thanks.

I did some homework on patching, and to my own surprice, the altered 
patch i have now even compiles on 9.0-RELEASE.  :D
only the param.h failed in the end, and i can not figure out why.

So i hope the change in that file is not that important.
I am running it now on a Supermicro 3U storage server with a few sata 
disk in it, with a single LSI 9211-8i card connected the the backplane.

It seems to do the job.
The driver coming with 9.0 had some trouble when i remirror some drives, 
i try that tomorrow.

i have put the patch here http://xs4all.nl/~doub/mps.patch maybe you can 
use it, maybe not.

my dmesg for what is worth

filer01# dmesg
Copyright (c) 1992-2012 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
         The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 9.0-RELEASE #1: Thu Jan 26 15:37:38 CET 2012
     root at filer01.neuteboom.local:/usr/obj/usr/src/sys/KRNL amd64
CPU: Intel(R) Xeon(R) CPU E31220 @ 3.10GHz (3093.04-MHz K8-class CPU)
   Origin = "GenuineIntel"  Id = 0x206a7  Family = 6  Model = 2a  
Stepping = 7

Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>

Features2=0x15bae3ff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,TSCDLT,XSAVE,AVX>
   AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
   AMD Features2=0x1<LAHF>
   TSC: P-state invariant, performance statistics
real memory  = 17179869184 (16384 MB)
avail memory = 16493346816 (15729 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <SUPERM SMCI--MB>
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s)
  cpu0 (BSP): APIC ID:  0
  cpu1 (AP): APIC ID:  2
  cpu2 (AP): APIC ID:  4
  cpu3 (AP): APIC ID:  6
ioapic0 <Version 2.0> irqs 0-23 on motherboard
kbd1 at kbdmux0
acpi0: <SUPERM SMCI--MB> on motherboard
acpi0: Power Button (fixed)
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
cpu2: <ACPI CPU> on acpi0
cpu3: <ACPI CPU> on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> irq 19 at device 6.0 on pci0
pci1: <ACPI PCI bus> on pcib1
mps0: <LSI SAS2008> port 0xe000-0xe0ff mem 0xfb600000-0xfb603fff irq 19 
at device 0.0 on pci1
mps0: Firmware: 11.00.00.00, Driver: 11.255.03.00-fbsd
mps0: IOCCapabilities: 
1285c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,HostDisc>
em0: <Intel(R) PRO/1000 Network Connection 7.2.3> port 0xf020-0xf03f mem 
0xfb800000-0xfb81ffff,0xfb824000-0xfb824fff irq 20 at device 25.0 on pci0
em0: No MSI/MSIX using a Legacy IRQ
em0: Ethernet address: 00:25:90:57:20:bd
ehci0: <EHCI (generic) USB 2.0 controller> mem 0xfb823000-0xfb8233ff irq 
16 at device 26.0 on pci0
usbus0: EHCI version 1.0
usbus0: <EHCI (generic) USB 2.0 controller> on ehci0
pcib2: <ACPI PCI-PCI bridge> irq 17 at device 28.0 on pci0
pci2: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> irq 17 at device 28.4 on pci0
pci3: <ACPI PCI bus> on pcib3
em1: <Intel(R) PRO/1000 Network Connection 7.2.3> port 0xd000-0xd01f mem 
0xfb700000-0xfb71ffff,0xfb720000-0xfb723fff irq 16 at device 0.0 on pci3
em1: MSIX: insufficient vectors, using MSI
em1: No MSI/MSIX using a Legacy IRQ
em1: Ethernet address: 00:25:90:57:20:bc
ehci1: <EHCI (generic) USB 2.0 controller> mem 0xfb822000-0xfb8223ff irq 
23 at device 29.0 on pci0
usbus1: EHCI version 1.0
usbus1: <EHCI (generic) USB 2.0 controller> on ehci1
pcib4: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci4: <ACPI PCI bus> on pcib4
vgapci0: <VGA-compatible display> mem 
0xf9000000-0xf9ffffff,0xfb000000-0xfb003fff,0xfa800000-0xfaffffff irq 23 
at device 3.0 on pci4
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
ahci0: <Intel Cougar Point AHCI SATA controller> port 
0xf070-0xf077,0xf060-0xf063,0xf050-0xf057,0xf040-0xf043,0xf000-0xf01f 
mem 0xfb821000-0xfb8217ff irq 19 at device 31.2 on pci0
ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich1: <AHCI channel> at channel 1 on ahci0
ahcich2: <AHCI channel> at channel 2 on ahci0
ahcich3: <AHCI channel> at channel 3 on ahci0
ahcich4: <AHCI channel> at channel 4 on ahci0
ahcich5: <AHCI channel> at channel 5 on ahci0
pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
acpi_button0: <Power Button> on acpi0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 950
Event timer "HPET" frequency 14318180 Hz quality 550
Event timer "HPET1" frequency 14318180 Hz quality 440
Event timer "HPET2" frequency 14318180 Hz quality 440
Event timer "HPET3" frequency 14318180 Hz quality 440
Event timer "HPET4" frequency 14318180 Hz quality 440
attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
Event timer "RTC" frequency 32768 Hz quality 0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
uart2: <16550 or compatible> port 0x3e8-0x3ef irq 10 on acpi0
orm0: <ISA Option ROM> at iomem 0xc0000-0xc7fff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
ppc0: cannot reserve I/O port range
est0: <Enhanced SpeedStep Frequency Control> on cpu0
p4tcc0: <CPU Frequency Thermal Control> on cpu0
est1: <Enhanced SpeedStep Frequency Control> on cpu1
p4tcc1: <CPU Frequency Thermal Control> on cpu1
est2: <Enhanced SpeedStep Frequency Control> on cpu2
p4tcc2: <CPU Frequency Thermal Control> on cpu2
est3: <Enhanced SpeedStep Frequency Control> on cpu3
p4tcc3: <CPU Frequency Thermal Control> on cpu3
ZFS filesystem version 5
ZFS storage pool version 28
Timecounters tick every 1.000 msec
usbus0: 480Mbps High Speed USB v2.0
usbus1: 480Mbps High Speed USB v2.0
ugen0.1: <Intel> at usbus0
uhub0: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus0
ugen1.1: <Intel> at usbus1
uhub1: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
da0 at mps0 bus 0 scbus0 target 8 lun 0
da0: <ATA WDC WD800JD-60LS 1D07> Fixed Direct Access SCSI-6 device
da0: 300.000MB/s transfers
da0: Command Queueing enabled
da0: 76319MB (156301488 512 byte sectors: 255H 63S/T 9729C)
da1 at mps0 bus 0 scbus0 target 9 lun 0
da1: <ATA WDC WD800JD-60LS 1D07> Fixed Direct Access SCSI-6 device
da1: 300.000MB/s transfers
da1: Command Queueing enabled
da1: 76319MB (156301488 512 byte sectors: 255H 63S/T 9729C)
da2 at mps0 bus 0 scbus0 target 10 lun 0
da2: <ATA WDC WD2500AAJS-6 2E25> Fixed Direct Access SCSI-6 device
da2: 300.000MB/s transfers
da2: Command Queueing enabled
da2: 238475MB (488397168 512 byte sectors: 255H 63S/T 30401C)
da3 at mps0 bus 0 scbus0 target 11 lun 0
da3: <ATA WDC WD2500JS-60M 2E04> Fixed Direct Access SCSI-6 device
da3: 300.000MB/s transfers
da3: Command Queueing enabled
da3: 238475MB (488397168 512 byte sectors: 255H 63S/T 30401C)
da4 at mps0 bus 0 scbus0 target 12 lun 0
da4: <ATA WDC WD2500JS-60M 2E04> Fixed Direct Access SCSI-6 device
da4: 300.000MB/s transfers
da4: Command Queueing enabled
da4: 238475MB (488397168 512 byte sectors: 255H 63S/T 30401C)
da5 at mps0 bus 0 scbus0 target 13 lun 0
da5: <ATA ST3250620NS BJH> Fixed Direct Access SCSI-6 device
da5: 150.000MB/s transfers
da5: Command Queueing enabled
da5: 238475MB (488397168 512 byte sectors: 255H 63S/T 30401C)
ses0 at mps0 bus 0 scbus0 target 16 lun 0
ses0: <LSI CORP SAS2X28 0717> Fixed Enclosure Services SCSI-5 device
ses0: 600.000MB/s transfers
ses0: Command Queueing enabled
ses0: SCSI-3 SES Device
cd0 at ahcich2 bus 0 scbus3 target 0 lun 0
SMP: AP CPU #1 Launched!
cd0: <TSSTcorp CDDVDW SN-208BB SB00> Removable CD-ROM SCSI-0 device
cd0: 150.000MB/s transfers (SATA 1.x, UDMA5, ATAPI 12bytes, PIO 8192bytes)
cd0: Attempt to query device size failed: NOT READY, Medium not present 
- tray closed
SMP: AP CPU #2 Launched!
SMP: AP CPU #3 Launched!
Timecounter "TSC-low" frequency 12082197 Hz quality 1000
da7 at mps0 bus 0 scbus0 target 15 lun 0
da7: <ATA GB0750C8047 HPG1> Fixed Direct Access SCSI-6 device
da7: 150.000MB/s transfers
da7: Command Queueing enabled
da7: 715404MB (1465149168 512 byte sectors: 255H 63S/T 91201C)
da6 at mps0 bus 0 scbus0 target 14 lun 0
da6: <ATA GB0750C8047 HPG1> Fixed Direct Access SCSI-6 device
da6: 150.000MB/s transfers
da6: Command Queueing enabled
da6: 715404MB (1465149168 512 byte sectors: 255H 63S/T 91201C)
Root mount waiting for: GMIRROR usbus1 usbus0
uhub0: 2 ports with 2 removable, self powered
uhub1: 2 ports with 2 removable, self powered
Root mount waiting for: GMIRROR usbus1 usbus0
ugen0.2: <vendor 0x8087> at usbus0
ugen1.2: <vendor 0x8087> at usbus1
uhub2: <vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr 2> 
on usbus0
uhub3: <vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr 2> 
on usbus1
uhub3: 6 ports with 6 removable, self powered
uhub2: 6 ports with 6 removable, self powered
ugen0.3: <Winbond Electronics Corp> at usbus0
ums0: <Winbond Electronics Corp Hermon USB hidmouse Device, class 0/0, 
rev 1.10/0.01, addr 3> on usbus0
ums0: 3 buttons and [Z] coordinates ID=0
ukbd0: <Winbond Electronics Corp Hermon USB hidmouse Device, class 0/0, 
rev 1.10/0.01, addr 3> on usbus0
kbd2 at ukbd0
Root mount waiting for: GMIRROR
Root mount waiting for: GMIRROR
GEOM_MIRROR: Force device gm0-p2 start due to timeout.
GEOM_MIRROR: Device mirror/gm0-p2 launched (1/2).
Trying to mount root from ufs:/dev/mirror/gm0-p2 [rw]...

Again, i am a complete novice with this sort of things, so forgive me 
any mistakes i made.

regards,
Johan Hendriks