mfi(4) lockups and the adapter event log
korvus at comcast.net
Mon Nov 9 15:48:55 UTC 2009
We saw an odd crash (more of a lockup) on one of our Dell's with a
PERC5/i RAID controller (running 6.3-RELEASE-p10):
mfi0: COMMAND 0xffffffff89dab0e8 TIMEOUT AFTER 31 SECONDS
(repeated many times with different commands)
mfi0: 3325 (310696326s/0x0020/4) - Type 18: Fatal firmware error: Line
1091 in ../../raid/verdeMain.c
After some troubleshooting with Dell, they determined that there was no
problem. An OpenManage live CD was used to run their diagnostic
utilities, which turned up nothing.
However, after rebooting the system and checking dmesg(8), it could be
seen that the adapter was doing some weird things during
initialization. It seemed to be going through the whole initialization
cycle (Firmware initialization started....) up to five times. This does
not jive with our other mfi/PERC5 systems. In addition to the multiple
init cycles it was also printing weeks worth of messages from the
adapter's event log (mostly battery relearns). Checking the event log
with linux-megacli showed TONS of messages. Trying to clear them using
linux-megacli seemed to cause a similar lockup, filled with command
timeouts, but no fatal firmware error.
We ended up clearing the event log using a linux live CD and the linux
native MegaCLI binaries. The system hasn't locked up again since, but
we're not sure what caused it in the first place.
Has anyone else seen any kind of oddities with the PERC series of
controllers and the adapter's event log? Does it just fill up after a
while and start to cause trouble? I seem to remember a similar post
recommending a cleaning of the event logs every so often.
More information about the freebsd-questions