Fatal Trap 12: page fault while in kernel mode on IBM x340 after Aug25CVS

[Robert Westendorp]

Hi,

We did an upgrade of one of our machines from an Aug 25 CVS and ever since
then one of the machines likes to reboot about once a day.

The machine is the following:
IBM x340
2x1GHZ Processors
1GB RAM
3x36GB Drives (RAID 5)
Mylex ExtreRAID 352

In the panic it mentions httpd, but I don't *think* it's apache (1.3.27)
that's causing the problem, or am I wrong? I also believe it is not a load
issue as the load does not get much about 2 most of the time.

The only special kernel config is that it's an SMP kernel and the following
options are set:

options         NMBCLUSTERS=20000
options         PMAP_SHPGPERPROC=600
options         IPFIREWALL
options         DUMMYNET

Below is a copy of the panic message I got from a serial console and below
that is a dmesg from today.

Fatal trap 12: page fault while in kernel mode
mp_lock = 01000002; cpuid = 1; lapic.id = 00000000
fault virtual address   = 0x13
fault code              = supervisor read, page not present
instruction pointer     = 0x8:0xc03151b8
stack pointer           = 0x10:0xe6b1cc28
frame pointer           = 0x10:0xe6b1cc30
code segment            = base 0x0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, def32 1, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 64499 (httpd)
interrupt mask          = net tty bio cam  <- SMP: XXX
trap number             = 12


Copyright (c) 1992-2003 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD 4.8-STABLE #5: Wed Sep  3 15:46:25 PDT 2003
    XX at xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Timecounter "i8254"  frequency 1193182 Hz
CPU: Intel Pentium III (996.85-MHz 686-class CPU)
  Origin = "GenuineIntel"  Id = 0x686  Stepping = 6

Features=0x383fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,
CMOV,PAT,PSE36,MMX,FXSR,SSE>
real memory  = 1207943168 (1179632K bytes)
avail memory = 1168994304 (1141596K bytes)
Programming 16 pins in IOAPIC #0
IOAPIC #0 intpin 2 -> irq 0
Programming 16 pins in IOAPIC #1
IOAPIC #1 intpin 10 -> irq 5
FreeBSD/SMP: Multiprocessor motherboard: 2 CPUs
 cpu0 (BSP): apic id:  1, version: 0x00040011, at 0xfee00000
 cpu1 (AP):  apic id:  0, version: 0x00040011, at 0xfee00000
 io0 (APIC): apic id: 14, version: 0x000f0011, at 0xfec00000
 io1 (APIC): apic id: 15, version: 0x000f0011, at 0xfec01000
Preloaded elf kernel "kernel" at 0xc04da000.
Pentium Pro MTRR support enabled
md0: Malloc disk
npx0: <math processor> on motherboard
npx0: INT 16 interface
pcib0: <ServerWorks NB6635 3.0LE host to PCI bridge> on motherboard
IOAPIC #1 intpin 11 -> irq 2
IOAPIC #1 intpin 0 -> irq 9
IOAPIC #1 intpin 2 -> irq 16
IOAPIC #0 intpin 9 -> irq 17
pci0: <PCI bus> on pcib0
pci0: <S3 Savage 4 graphics accelerator> at 1.0
pcn0: <AMD PCnet/PCI 10/100BaseTX> port 0x2000-0x201f mem
0xfeb7fc00-0xfeb7fc1f irq 2 at device 2.0 on pci0
pcn0: Ethernet address: 00:02:55:91:a4:ad
miibus0: <MII bus> on pcn0
ukphy0: <Generic IEEE 802.3u media interface> on miibus0
ukphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
fxp0: <Intel 82550 Pro/100 Ethernet> port 0x2040-0x207f mem
0xfeb40000-0xfeb5ffff,0xfeb7e000-0xfeb7efff irq 9 at d
evice 9.0 on pci0
fxp0: Ethernet address 00:02:b3:62:68:d3
inphy0: <i82555 10/100 media interface> on miibus1
inphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
fxp1: <Intel 82550 Pro/100 Ethernet> port 0x2080-0x20bf mem
0xfeb20000-0xfeb3ffff,0xfeb7d000-0xfeb7dfff irq 16 at
device 10.0 on pci0
fxp1: Ethernet address 00:02:b3:62:6a:19
inphy1: <i82555 10/100 media interface> on miibus2
inphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
isab0: <ServerWorks IB6566 PCI to ISA bridge> at device 15.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <ServerWorks ROSB4 ATA33 controller> port 0x840-0x84f at device
15.1 on pci0
ata0: at 0x1f0 irq 14 on atapci0
ata1: at 0x170 irq 15 on atapci0
ohci0: <OHCI (generic) USB controller> mem 0xff700000-0xff700fff irq 17 at
device 15.2 on pci0
usb0: OHCI version 1.0, legacy support
usb0: <OHCI (generic) USB controller> on ohci0
usb0: USB revision 1.0
uhub0: (0x1166) OHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 4 ports with 4 removable, self powered
pcib1: <ServerWorks NB6635 3.0LE host to PCI bridge> on motherboard
IOAPIC #1 intpin 4 -> irq 18
pci1: <PCI bus> on pcib1
pcib2: <PCI to PCI bridge (vendor=8086 device=0964)> at device 5.0 on pci1
pci2: <PCI bus> on pcib2
mly0: <Mylex AcceleRAID 352> mem 0xedffe000-0xedffffff irq 18 at device 5.1
on pci1
mly0: AcceleRAID 352  , 2 channels, firmware 6.00-1-00 (20000426), 64MB RAM
orm0: <Option ROMs> at iomem
0xc0000-0xc9fff,0xca000-0xcb7ff,0xcb800-0xccfff,0xcd000-0xcffff on isa0
pmtimer0 on isa0
fdc0: <NEC 72065B or clone> at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0
fdc0: FIFO enabled, 8 bytes threshold
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> flags 0x1 irq 1 on atkbdc0
kbd0 at atkbd0
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x100>
sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0
sio0: type 16550A, console
sio1 at port 0x2f8-0x2ff irq 3 on isa0
sio1: type 16550A
ppc0: <Parallel port> at port 0x378-0x37f irq 7 on isa0
ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode
plip0: <PLIP network interface> on ppbus0
lpt0: <Printer> on ppbus0
lpt0: Interrupt-driven port
ppi0: <Parallel I/O> on ppbus0
APIC_IO: Testing 8254 interrupt delivery
APIC_IO: Broken MP table detected: 8254 is not connected to IOAPIC #0 intpin
2
APIC_IO: routing 8254 via 8259 and IOAPIC #0 intpin 0
DUMMYNET initialized (011031)
IP packet filtering initialized, divert disabled, rule-based forwarding
enabled, default to deny, logging disabled
SMP: AP CPU #1 Launched!
acd0: CDROM <LG CD-ROM CRN-8245B> at ata0-master PIO4
da0 at mly0 bus 2 target 0 lun 0
da0: <RAID 5 online > Fixed Direct Access SCSI-2 device
da0: 135.168MB/s transfers
da0: 68664MB (140623872 512 byte sectors: 255H 63S/T 8753C)
Mounting root from ufs:/dev/da0s1a


---------------------------------------------------------------
Robert Westendorp
Systems Administrator
rjwsys at uniserve.com
---------------------------------------------------------------
UNIServe Online Disclaimer:
"The sender intends this message for a specific recipient and, as it may
contain information that is privileged or confidential, any use,
dissemination, forwarding, or copying by anyone without permission from the
sender is prohibited. Personal e-mail may contain views that are not
necessarily those of the company".