i386/126866: isp(4) - kernel panic on card initialization

[Ross]

>Number:         126866
>Category:       i386
>Synopsis:       isp(4) - kernel panic on card initialization
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-i386
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue Aug 26 20:20:02 UTC 2008
>Closed-Date:
>Last-Modified:
>Originator:     Ross
>Release:        Freebsd 7.0
>Organization:
>Environment:
FreeBSD <servername> 7.0-RELEASE-p3 FreeBSD 7.0-RELEASE-p3 #22: Tue Aug 26 13:18:24 EDT 2008     <servername>:/usr/obj/usr/src/sys/BL460C  i386
>Description:
isp(4) driver can crash the kernel with a fatal trap 12/Page fault upon initialization of the driver - this is generally evident in a boot-off-SAN configuration where the card is already active upon boot.

Debugging shows the last outputted message as "isp1: Port Database Changed: freeze simq (loopdown)" [line 288 in isp_freebsd.c].  Further added debugging shows that the function isp_freeze_loopdown() will never return, pointing the problem most likely being with the line 'xpt_freeze_simq(isp->isp_sim, 1);' [line 290 in isp_freebsd.c]

Normal startup/activation of the driver will occur about 50% of the time, which seems to point to the driver receiving a FC command/notification before it's ready for it.

-= Normal successful startup: [hint.isp.0.debug = 0x11F]
..
Aug 26 15:45:16 controller kernel: isp0: <Qlogic ISP 2432 PCI FC-AL Adapter> port 0x4000-0x40ff mem 0xfdff0000-0xfdff3fff irq 18 at device 0.0 on pci16
Aug 26 15:45:16 controller kernel: isp0: set PCI latency to 64
Aug 26 15:45:16 controller kernel: isp0: [ITHREAD]
Aug 26 15:45:16 controller kernel: isp0: Board Type 2422, Chip Revision 0x2, resident F/W Revision 4.0.90
Aug 26 15:45:16 controller kernel: isp0: 2K Logins Supported
Aug 26 15:45:16 controller kernel: isp0: Last F/W revision was 4.0.90
Aug 26 15:45:16 controller kernel: isp0: 4096 max I/O command limit set
Aug 26 15:45:16 controller kernel: isp0: line 1207: markportdb
Aug 26 15:45:16 controller kernel: isp0: Starting Initial Loop Down Timer
Aug 26 15:45:16 controller kernel: isp1: <Qlogic ISP 2432 PCI FC-AL Adapter> port 0x4400-0x44ff mem 0xfdfe0000-0xfdfe3fff irq 19 at device 0.1 on pci16
Aug 26 15:45:16 controller kernel: isp1: set PCI latency to 64
Aug 26 15:45:16 controller kernel: isp1: [ITHREAD]
Aug 26 15:45:16 controller kernel: isp1: Board Type 2422, Chip Revision 0x2, resident F/W Revision 4.0.90
Aug 26 15:45:16 controller kernel: isp1: 2K Logins Supported
Aug 26 15:45:16 controller kernel: isp1: Last F/W revision was 4.0.90
Aug 26 15:45:16 controller kernel: isp1: 4096 max I/O command limit set
Aug 26 15:45:16 controller kernel: isp1: line 1207: markportdb
Aug 26 15:45:16 controller kernel: isp1: Starting Initial Loop Down Timer
..
-=

-= Failed startup
Aug 26 15:45:16 controller kernel: isp0: <Qlogic ISP 2432 PCI FC-AL Adapter> port 0x4000-0x40ff mem 0xfdff0000-0xfdff3fff irq 18 at device 0.0 on pci16
Aug 26 15:45:16 controller kernel: isp0: set PCI latency to 64
Aug 26 15:45:16 controller kernel: isp0: [ITHREAD]
Aug 26 15:45:16 controller kernel: isp0: Board Type 2422, Chip Revision 0x2, resident F/W Revision 4.0.90
Aug 26 15:45:16 controller kernel: isp0: 2K Logins Supported
Aug 26 15:45:16 controller kernel: isp0: Last F/W revision was 4.0.90
Aug 26 15:45:16 controller kernel: isp0: 4096 max I/O command limit set
Aug 26 15:45:16 controller kernel: isp0: line 1207: markportdb
Aug 26 15:45:16 controller kernel: isp0: Starting Initial Loop Down Timer
Aug 26 15:45:16 controller kernel: isp1: <Qlogic ISP 2432 PCI FC-AL Adapter> port 0x4400-0x44ff mem 0xfdfe0000-0xfdfe3fff irq 19 at device 0.1 on pci16
Aug 26 15:45:16 controller kernel: isp1: set PCI latency to 64
Aug 26 15:45:16 controller kernel: isp1: [ITHREAD]
Aug 26 15:45:16 controller kernel: isp1: line 5345: markportdb
Aug 26 15:45:16 controller kernel: isp1: Port Database Changed
Aug 26 15:45:16 controller kernel: isp1: Port Database Changed: freeze simq (loopdown)
[point of crash]
Fatal Trap 12: page fault while in kernel
fault code = Supervisor read, page not present
instruction pointer:    0x20 : 0xc0243a46
stack pointer:          0x20 : 0xc0af63cc
frame pointer:          0x20 : 0xc0af63cc
..
-=


>How-To-Repeat:

Reboot the box, and hope the FC command doesn't come down the pipe before the driver is fully initialized.  :-)
>Fix:
none.

>Release-Note:
>Audit-Trail:
>Unformatted: