6.2R-p3 and crash dump

Krzysztof Cholewa kcholewa at freebsd.lublin.pl
Fri May 18 07:16:11 UTC 2007


Hello.
I've got a box with FreeBSD 6.2Rp3 witch working as e router(quagga -
RIP, BGPD) and NAT(ipnat) and if_vlan load from modul. Kernel is GENERIC
with few options as POLLING and QUOTA. Last week crashed 3 times with
the same crash dump:

kgdb: kvm_nlist(_stopped_cpus): 
kgdb: kvm_nlist(_stoppcbs): 
[GDB will not be able to debug user-mode
threads: /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"]
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you
are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for
details.
This GDB was configured as "i386-marcel-freebsd".

Unread portion of the kernel message buffer:


Fatal trap 12: page fault while in kernel mode
fault virtual address   = 0xc
fault code              = supervisor read, page not present
instruction pointer     = 0x20:0xc06abbc8
stack pointer           = 0x28:0xe4fa89f8
frame pointer           = 0x28:0xe4fa8a04
code segment            = base 0x0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, def32 1, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 11 (swi1: net)
trap number             = 12
panic: page fault
Uptime: 4d2h24m9s
Dumping 2047 MB (2 chunks)
  chunk 0: 1MB (157 pages) ... ok
  chunk 1: 2047MB (523980 pages) 2031 2015 1999 1983 1967 1951 1935 1919
1903 1887 1871 1855 1839 1823 1807 1791 1775 1759 1743 1727 1711 1695
1679 1663 1647 1631 1615 1599 1583 1567 1551 1535 1519 1503 1487 1471
1455 1439 1423 1407 1391 1375 1359 1343 1327 1311 1295 1279 1263 1247
1231 1215 1199 1183 1167 1151 1135 1119 1103 1087 1071 1055 1039 1023
1007 991 975 959 943 927 911 895 879 863 847 831 815 799 783 767 751 735
719 703 687 671 655 639 623 607 591 575 559 543 527 511 495 479 463 447
431 415 399 383 367 351 335 319 303 287 271 255 239 223 207 191 175 159
143 127 111 95 79 63 47 31 15

#0  doadump () at pcpu.h:165
165             __asm __volatile("movl %%fs:0,%0" : "=r" (td));
(kgdb) bt
#0  doadump () at pcpu.h:165
#1  0xc0675a56 in boot (howto=260)
at /usr/src/sys/kern/kern_shutdown.c:409
#2  0xc0675cec in panic (fmt=0xc08e07ab "%s")
at /usr/src/sys/kern/kern_shutdown.c:565
#3  0xc088ab94 in trap_fatal (frame=0xe4fa89b8, eva=12)
at /usr/src/sys/i386/i386/trap.c:837
#4  0xc088a8fb in trap_pfault (frame=0xe4fa89b8, usermode=0, eva=12)
at /usr/src/sys/i386/i386/trap.c:745
#5  0xc088a539 in trap (frame=
      {tf_fs = -963313656, tf_es = -963248088, tf_ds = 40, tf_edi = 1,
tf_esi = 0, tf_ebp = -453342716, tf_isp = -453342748, tf_ebx = 57,
tf_edx = 0, tf_ecx = 0, tf_eax = 70393375, tf_trapno = 12, tf_err = 0,
tf_eip = -1066746936, tf_cs = 32, tf_eflags = 590338, tf_esp =
-893243712, tf_ss = -893243712}) at /usr/src/sys/i386/i386/trap.c:435
#6  0xc0878c0a in calltrap () at /usr/src/sys/i386/i386/exception.s:139
#7  0xc06abbc8 in m_copydata (m=0x0, off=0, len=1, cp=0xcac22f08 "3).\r
\n250 CWD command successful\r\n250 CWD command*#")
    at /usr/src/sys/kern/uipc_mbuf.c:543
#8  0xc695c218 in ?? ()
#9  0xc6a28900 in ?? ()
#10 0x000000a0 in ?? ()
#11 0x0000003a in ?? ()
#12 0xcac22f08 in ?? ()
#13 0x1b540000 in ?? ()
#14 0xe4fa8a7c in ?? ()
#15 0xa4607bbb in ?? ()
#16 0xc6a28900 in ?? ()
#17 0xc6901010 in ?? ()
#18 0xc6901024 in ?? ()
#19 0xcac22e08 in ?? ()
#20 0xcac22ea0 in ?? ()
#21 0xffffffc6 in ?? ()
#22 0x00000000 in ?? ()
#23 0x00000001 in ?? ()
#24 0xffffffff in ?? ()
#25 0x000000a0 in ?? ()
#26 0x0000003a in ?? ()
#27 0x0000003a in ?? ()
#28 0xcac22e00 in ?? ()
#29 0xcd074a00 in ?? ()
#30 0xe4fa8b68 in ?? ()
#31 0xe4fa8a88 in ?? ()
#32 0xc695c73a in ?? ()
#33 0xe4fa8b68 in ?? ()
#34 0xcd074a00 in ?? ()
#35 0xcac22e00 in ?? ()
#36 0x00000001 in ?? ()
#37 0x000000c6 in ?? ()
#38 0xd15fd880 in ?? ()
#39 0xcd074a00 in ?? ()
#40 0xe4fa8ad4 in ?? ()
#41 0xc6960bfd in ?? ()
#42 0xe4fa8b68 in ?? ()
#43 0xd15fd880 in ?? ()
#44 0xcd074a00 in ?? ()
#45 0x00000020 in ?? ()
#46 0xc680d300 in ?? ()
#47 0xc64a8000 in ?? ()
#48 0xc64aa000 in ?? ()
#49 0xe4fa8adc in ?? ()
#50 0xc0506b47 in bge_start_locked (ifp=0xcac22ec0)
at /usr/src/sys/dev/bge/if_bge.c:3029
Previous frame inner to this frame (corrupt stack?)
(kgdb) up 7
#7  0xc06abbc8 in m_copydata (m=0x0, off=0, len=1, cp=0xcac22f08 "3).\r
\n250 CWD command successful\r\n250 CWD command*#")
    at /usr/src/sys/kern/uipc_mbuf.c:543
543                     m = m->m_next;
(kgdb) list *0xc06abbc8
0xc06abbc8 is in m_copydata (libkern.h:56).
51      static __inline int imax(int a, int b) { return (a > b ? a :
b); }
52      static __inline int imin(int a, int b) { return (a < b ? a :
b); }
53      static __inline long lmax(long a, long b) { return (a > b ? a :
b); }
54      static __inline long lmin(long a, long b) { return (a < b ? a :
b); }
55      static __inline u_int max(u_int a, u_int b) { return (a > b ?
a : b); }
56      static __inline u_int min(u_int a, u_int b) { return (a < b ?
a : b); }
57      static __inline quad_t qmax(quad_t a, quad_t b) { return (a >
b ? a : b); }
58      static __inline quad_t qmin(quad_t a, quad_t b) { return (a <
b ? a : b); }
59      static __inline u_long ulmax(u_long a, u_long b) { return (a >
b ? a : b); }
60      static __inline u_long ulmin(u_long a, u_long b) { return (a <
b ? a : b); }

The box is IBM x346:
bge0 at pci5:0:0:  class=0x020000 card=0x02c61014 chip=0x165914e4 rev=0x11
hdr=0x00
    vendor   = 'Broadcom Corporation'
    device   = 'BCM5750A1 NetXtreme Gigabit Ethernet PCI Express'
    class    = network
    subclass = ethernet
bge1 at pci6:0:0:  class=0x020000 card=0x02c61014 chip=0x165914e4 rev=0x11
hdr=0x00
    vendor   = 'Broadcom Corporation'
    device   = 'BCM5750A1 NetXtreme Gigabit Ethernet PCI Express'
    class    = network
    subclass = ethernet

Copyright (c) 1992-2007 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights
reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 6.2-RELEASE-p3 #0: Fri Apr 20 13:11:37 UTC 2007
    root@:/usr/obj/usr/src/sys/ITNET-BRZEG
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Xeon(TM) CPU 3.80GHz (3800.16-MHz 686-class CPU)
  Origin = "GenuineIntel"  Id = 0xf43  Stepping = 3

Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x659d<SSE3,RSVD2,MON,DS_CPL,EST,TM2,CNTX-ID,CX16,<b14>>
  AMD Features=0x20000000<LM>
  Logical CPUs per core: 2
real memory  = 2147270656 (2047 MB)
avail memory = 2096324608 (1999 MB)
MPTable: <IBM ENSW x346 SMP    >
ioapic0: Assuming intbase of 0
ioapic1: Assuming intbase of 24
ioapic2: Assuming intbase of 48
ioapic3: Assuming intbase of 72
ioapic4: Assuming intbase of 96
ioapic4 <Version 2.0> irqs 96-119 on motherboard
ioapic3 <Version 2.0> irqs 72-95 on motherboard
ioapic2 <Version 2.0> irqs 48-71 on motherboard
ioapic1 <Version 2.0> irqs 24-47 on motherboard
ioapic0 <Version 2.0> irqs 0-23 on motherboard
kbd1 at kbdmux0
ath_hal: 0.9.17.2 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413,
RF5413)
cpu0 on motherboard
pcib0: <MPTable Host-PCI bridge> pcibus 0 on motherboard
pci0: <PCI bus> on pcib0
pci0: <unknown> at device 0.1 (no driver attached)
pcib1: <PCI-PCI bridge> at device 2.0 on pci0
pci2: <PCI bus> on pcib1
pcib2: <PCI-PCI bridge> at device 0.0 on pci2
pci3: <PCI bus> on pcib2
pcib3: <PCI-PCI bridge> at device 0.2 on pci2
pci4: <PCI bus> on pcib3
pcib4: <MPTable PCI-PCI bridge> at device 4.0 on pci0
pci5: <PCI bus> on pcib4
bge0: <Broadcom BCM5750 B1, ASIC rev. 0x4101> mem 0xdeff0000-0xdeffffff
irq 16 at device 0.0 on pci5
miibus0: <MII bus> on bge0
brgphy0: <BCM5750 10/100/1000baseTX PHY> on miibus0
brgphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX,
1000baseTX-FDX, auto
bge0: Ethernet address: 00:14:5e:ac:9c:41
pcib5: <MPTable PCI-PCI bridge> at device 5.0 on pci0
pci6: <PCI bus> on pcib5
bge1: <Broadcom BCM5750 B1, ASIC rev. 0x4101> mem 0xdcff0000-0xdcffffff
irq 16 at device 0.0 on pci6
miibus1: <MII bus> on bge1
brgphy1: <BCM5750 10/100/1000baseTX PHY> on miibus1
brgphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX,
1000baseTX-FDX, auto
bge1: Ethernet address: 00:14:5e:ac:9c:40
pcib6: <PCI-PCI bridge> at device 6.0 on pci0
pci7: <PCI bus> on pcib6
pcib7: <MPTable PCI-PCI bridge> at device 0.0 on pci7
pci8: <PCI bus> on pcib7
ahd0: <Adaptec AIC7902 Ultra320 SCSI adapter> port
0x4000-0x40ff,0x4100-0x41ff mem 0xdaffe000-0xdaffffff irq 27 at device
7.0 on pci8
ahd0: [GIANT-LOCKED]
aic7902: Ultra320 Wide Channel A, SCSI Id=7, PCI-X 101-133Mhz, 512 SCBs
ahd1: <Adaptec AIC7902 Ultra320 SCSI adapter> port
0x4200-0x42ff,0x4300-0x43ff mem 0xdaffc000-0xdaffdfff irq 24 at device
7.1 on pci8
ahd1: [GIANT-LOCKED]
aic7902: Ultra320 Wide Channel B, SCSI Id=7, PCI-X 101-133Mhz, 512 SCBs
pcib8: <PCI-PCI bridge> at device 0.2 on pci7
pci9: <PCI bus> on pcib8
pci0: <base peripheral> at device 8.0 (no driver attached)
uhci0: <Intel 82801EB (ICH5) USB controller USB-A> port 0x2200-0x221f
irq 16 at device 29.0 on pci0
uhci0: [GIANT-LOCKED]
usb0: <Intel 82801EB (ICH5) USB controller USB-A> on uhci0
usb0: USB revision 1.0
uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 2 ports with 2 removable, self powered
uhci1: <Intel 82801EB (ICH5) USB controller USB-B> port 0x2600-0x261f
irq 19 at device 29.1 on pci0
uhci1: [GIANT-LOCKED]
usb1: <Intel 82801EB (ICH5) USB controller USB-B> on uhci1
usb1: USB revision 1.0
uhub1: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub1: 2 ports with 2 removable, self powered
ehci0: <Intel 82801EB/R (ICH5) USB 2.0 controller> mem
0xf9000000-0xf90003ff irq 23 at device 29.7 on pci0
ehci0: [GIANT-LOCKED]
usb2: EHCI version 1.0
usb2: companion controllers, 2 ports each: usb0 usb1
usb2: <Intel 82801EB/R (ICH5) USB 2.0 controller> on ehci0
usb2: USB revision 2.0
uhub2: Intel EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
uhub2: 4 ports with 4 removable, self powered
pcib9: <MPTable PCI-PCI bridge> at device 30.0 on pci0
pci1: <PCI bus> on pcib9
pci1: <display, VGA> at device 6.0 (no driver attached)
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel ICH5 UDMA100 controller> port
0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x480-0x48f at device 31.1 on pci0
ata0: <ATA channel 0> on atapci0
ata1: <ATA channel 1> on atapci0
pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
pmtimer0 on isa0
orm0: <ISA Option ROM> at iomem 0xc0000-0xcafff on isa0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: model Generic PS/2 mouse, device ID 0
fdc0: <Enhanced floppy controller> at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2
on isa0
fdc0: [FAST]
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
ppc0: parallel port not found.
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0
sio0: type 16550A
sio1: configured irq 3 not in bitmap of probed irqs 0
sio1: port may not be enabled
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on
isa0
unknown: <PNP0c01> can't assign resources (memory)
unknown: <PNP0303> can't assign resources (port)
Timecounter "TSC" frequency 3800161586 Hz quality 800
Timecounters tick every 1.000 msec
acd0: CDRW <UJDA770 DVD/CDRW/1.20> at ata0-master UDMA33
Waiting 5 seconds for SCSI devices to settle
ses0 at ahd1 bus 0 target 8 lun 0
ses0: <IBM 40K6480a S320  0 1> Fixed Processor SCSI-2 device 
ses0: 3.300MB/s transfers
ses0: SAF-TE Compliant Device
da0 at ahd1 bus 0 target 0 lun 0
da0: <IBM-ESXS MAN3184MC     F 5408> Fixed Direct Access SCSI-3 device 
da0: 160.000MB/s transfers (80.000MHz, offset 127, 16bit), Tagged
Queueing Enabled
da0: 17357MB (35548320 512 byte sectors: 255H 63S/T 2212C)
Trying to mount root from ufs:/dev/da0s1a
WARNING: / was not properly dismounted
WARNING: /mnt/big was not properly dismounted
wan0: link state changed to DOWN
lan0: link state changed to DOWN
IP Filter: v4.1.13 initialized.  Default = pass all, Logging = enabled
lan0: link state changed to UP
vlan6: link state changed to UP
vlan5: link state changed to UP
wan0: link state changed to UP
vlan50: link state changed to UP
vlan4: link state changed to UP


I'm not programmer and I don't understand debugger.

btw. forgive me my english. ;)

best regards.

-- 
Krzysztof Cholewa


More information about the freebsd-net mailing list