Bug Report: IBM x3650M4 (32GB, 2x4-core Xeon E5-2600, IBM ServeRaid M5110e): fails in install with NMI

[Peter Wemm]

On Tue, Aug 28, 2012 at 9:38 AM, John Baldwin <jhb at freebsd.org> wrote:
> On Tuesday, August 28, 2012 10:05:07 am Mike A wrote:
>> On Tue, Aug 28, 2012 at 09:34:14AM -0400, John Baldwin wrote:
>> > On Monday, August 27, 2012 4:38:17 pm Mike A wrote:
>> > > IBM x3650M4 (32GB, 2x4-core Xeon E5-2600, IBM ServeRaid M5110e)
>> > >
>> > > I just got handed 4 of the subject boxes with instructions "put 'em to
>> > > work". Naturally I tried FreeBSD first, on one of the machines. Boot from
>> > > the 9.0 AMD64 boot-only install CD fails. Things look fine until the last
>> > > several lines of the (verbose enabled) boot sequence, which (from an
>> > > insufficiently-wide phone camera capture) are:
>> > >
>> > > "
>> > > mpt0: <LSILogic SAS/SATA Adapter> port 0x3000-0x[lost off right edge of
>> > phone]
>> > > xc5d00000-0xc5deffff irq 34 at device 0.0 on pci[lost]
>> > > mpt0: attempting to allocate 1 MSI vectors (1 su[lost]
>> > > msi: routing MSI IRQ 256 to local APIC 0 vector [lost]
>> > > mpt0: using IRQ 256 for MSI
>> > > mpt0: soft reset failed, device not running
>> > > NMI ISA 2c, EISA 0
>> > > NMI ... going to debugger
>> > > mpt0: hard reset failed
>> > > "
>> >
>> > Does setting 'hint.mpt.0.msi_enable=0' in the loader make a difference?
>>
>> Thanks VERY MUCH (and come collect your steak dinner at Cattlemen's Cafe in
>> OKC, next time you're in the area) for the very quick response.
>>
>> I will be happy to try that, but need guidance. This is an install from
>> CD (burned from FreeBSD-9.1-RC1-amd64-disc1.iso), and I don't know how to
>> insert a loader hint in that process.
>
> When the loader menu pops up, choose the "escape to loader prompt" option,
> then type 'set hint.mpt.0.msi_enable=0' followed by 'boot'.  There's no
> guarantee this will help, btw, just something to try out first.
>
> If that doesn't work, you can also try setting 'machdep.kdb_on_nmi=0' using
> the same trick.
>
> If that still doesn't help, please boot another OS that does and get the
> output of 'lspci -v' or 'pciconf -lvb' or equivalent so we can see exactly
> which mpt adapter it is.  I think there is one class of mpt(4) cards that
> we do not yet support properly.  Ah, yes, this PR:
>
> http://www.freebsd.org/cgi/query-pr.cgi?pr=149220
>
> I think this may in fact be your adapter.  This was fixed after 9.0, so try
> a 9.1-RC1 install disk instead and see if it works better.

Is it actually an mpt?  There's a number of current servers that have
*mfi* raid controllers that are mis-identified as mpt and being
claimed by the mpt driver.  Naturally this does not work well.  I
recognize the exact failure text from a failure we had in the
freebsd.org cluster a few days ago with 9.0-RELEASE.

The good news is that 9-STABLE or 9.1-RC get it right, at least on our
hardware. It correctly attaches as "mpt".

The bad news is that *our* particular servers have newer broadcom
chips in them and none of the 9.x releases or 9.1-RC's work on it, at
all.  That's probably not relevant to the original reporter though.

-- 
Peter Wemm - peter at wemm.org; peter at FreeBSD.org; peter at yahoo-inc.com; KI6FJV
"All of this is for nothing if we don't go to the stars" - JMS/B5
"If Java had true garbage collection, most programs would delete
themselves upon execution." -- Robert Sewell