ips(4) in toaster mode FreeBSD 7.2
jps at semihalf.com
Mon Dec 28 16:46:11 UTC 2009
I'm writing to you because I've seen that you have recently commited patches to
the FreeBSD ips(4) driver and perhaps you can shed some light on a problem I've
encountered. Here is a description:
Recently on one of our servers (IBM xSeries 345 [8760 M1X] with IBM ServeRAID 5i
II (Sarasota) RAID controller) ips driver threw a warning about timed-out
command and adapter being in toaster mode. After that kernel paniced - see
console message below:
=== Begin of console message ===
ips0: WARNING: command timeout. Adapter is in toaster mode, resetting to known state
ips: io error, status=0x2000c
ipsd0: iobuf error 5
ips0: resetting adaptegr_,v ftsh_idso nmea(y) :tiapksed 0usp1 ft[oW R5I
= 1543241728, length=16384)]error = 5
ips0: syncing config
Sleeping thread (tid 100006, pid 15) owns a non-sleepable lock
panic: sleeping thread
cpuid = 2
=== End of console message ===
Lines 5 to 7 are two kernel messages mixed together. They say something like this:
ips0: resetting adapter, this may take up to 5 minutes
g_vfs_done(): ipsd0s1f[WRITE(offset=154321728, length=16384)]error = 5
After displaying the above messages system is completely unresponsive. The only
solution is to reboot.
Messages come from functions located in files:
"WARNING: [...]": ips_timeout(): sys/dev/ips/ips.c;
"resetting adapter, [...]": ips_morpheus_reinit(): sys/dev/ips/ips.c;
"syncing config": ips_clear_adapter(): sys/dev/ips/ips_commands.c;
I have found someone reporting similar problem (ips in toaster mode
throwing a warning and kernel panic after that) to freebsd-stable list in Nov
The difference is that our server was almost idle (Christmas time) compared to
the situation described in the above thread (heavy disk usage during backups).
I've checked controller status with IBM's tools (IBM ServeRAID Manager) and it's
OK. /var/log/messages yields nothing that could lead to problem's
explanation. Server is now up and running, but the reason for this panic is
still unclear. I'd be grateful for hints.
Also I'd like to know if there are any new changes to be commited to ips driver
in future. If that's the case then I will wait for them before applying recent
changes to our system.
Some info about the system:
# uname -a
FreeBSD xxx.xxx.xxx 7.2-RELEASE-p4 FreeBSD 7.2-RELEASE-p4 #0: Thu Oct 22
11:01:23 CEST 2009 xxx at xxx.xxx.xxx:/usr/obj/usr/src/sys/XSERIES345 i386
# pciconf -lcv
ips0 at pci0:8:2:0: class=0x010400 card=0x02591014 chip=0x01bd1014 rev=0x00
vendor = 'Elektronik'
device = 'ServeRAID 4/5 Morpheus SCSI RAID Controller'
class = mass storage
subclass = RAID
cap 01 = powerspec 2 supports D0 D3 current D0
ServeRAID BIOS/Firmware version: 7.12.02
Kernel config is a GENERIC config without unnecessary drivers and features.
I'm able to provide any further information about the system if needed.
I also send this e-mail to freebsd-stable with hope that somebody has some ideas
on my problem.
More information about the freebsd-stable