amd64/150367: Possible QLogic fiber channel regression in 8.1-RELEASE (and above)

[Mahlon E. Smith]

>Number:         150367
>Category:       amd64
>Synopsis:       Possible QLogic fiber channel regression in 8.1-RELEASE (and above)
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    freebsd-amd64
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue Sep 07 21:40:06 UTC 2010
>Closed-Date:
>Last-Modified:
>Originator:     Mahlon E. Smith
>Release:        8.1-RELEASE
>Organization:
LAIKA, Inc.
>Environment:
Tested on 8.1-RELEASE amd64, and 8.1-STABLE amd64 (from September 3rd, 2010.) GENERIC kernel.
>Description:

I've got a pair of QLogic 2312 cards, that are identified to the kernel as
such:

    isp0: <Qlogic ISP 2312 PCI FC-AL Adapter> port 0xec00-0xecff mem 0xfc6ff000-0xfc6fffff irq 160 at device 3.0 on pci15
    isp1: <Qlogic ISP 2312 PCI FC-AL Adapter> port 0xe800-0xe8ff mem 0xfc6fe000-0xfc6fefff irq 161 at device 3.1 on pci15
    isp2: <Qlogic ISP 2312 PCI FC-AL Adapter> port 0xdc00-0xdcff mem 0xfc4ff000-0xfc4fffff irq 224 at device 4.0 on pci16
    isp3: <Qlogic ISP 2312 PCI FC-AL Adapter> port 0xd800-0xd8ff mem 0xfc4fe000-0xfc4fefff irq 225 at device 4.1 on pci16

They are directly attached to a large disk array.  Under FreeBSD
8.0-RELEASE, all devices appear as expected during the boot process
(or during a 'camcontrol rescan all'), and are under fairly heavy use
without issue.

I upgraded to 8.1-RELEASE (for ZFSv14), and the devices no longer
function.  Tried an 8.1-STABLE with the same result.  Because I needed
to have this setup operational again quickly, I opted to downgrade back
to 8.0-RELEASE, and the problem disappeared.

The errors during boot were:

    isp3: Mailbox Command 'ABORT' failed (COMMAND PARAMETER ERROR)
    isp3: isp_watchdog: timeout for handle 0x5e205e

In the interest of brevity, I've omitted identical errors from the other
isp devices.  These messages flood the screen from all isp devices, and
the machine is unresponsive until the FC cards are unplugged.

I am using the ispfw module, though I'm not honestly sure it matters,
since that particular card isn't listed in the registered firmware:

    ispfw: registered firmware <isp_1040>
    ispfw: registered firmware <isp_1040_it>
    ispfw: registered firmware <isp_1080>
    ispfw: registered firmware <isp_1080_it>
    ispfw: registered firmware <isp_12160>
    ispfw: registered firmware <isp_12160_it>
    ispfw: registered firmware <isp_2100>
    ispfw: registered firmware <isp_2200>
    ispfw: registered firmware <isp_2300>
    ispfw: registered firmware <isp_2322>
    ispfw: registered firmware <isp_2400>
    ispfw: registered firmware <isp_2400_multi>
    ispfw: registered firmware <isp_2500>
    ispfw: registered firmware <isp_2500_multi>

I'm happy to discuss the other hardware on this machine offline if it                                                                                                                                                                            ´
helps -- I unfortunately am not currently able to use this environment
as a tester, however.  (That may be a possibility at a later date, in
the coming months.)


>How-To-Repeat:
Simply booting into 8.1 or higher was enough to cause the timeouts.
>Fix:
No fix known, and without the luxury of time to properly diagnose, I downgraded to 8.0-RELEASE.  Everything came up fine.

>Release-Note:
>Audit-Trail:
>Unformatted: