misc/74909: SMP crash FreeBSD-5.3-p2 (swi1: net) / ipllog
Geoffrey Lassner
freebsd53 at zyni.com
Thu Dec 9 20:40:23 PST 2004
>Number: 74909
>Category: misc
>Synopsis: SMP crash FreeBSD-5.3-p2 (swi1: net) / ipllog
>Confidential: no
>Severity: critical
>Priority: medium
>Responsible: freebsd-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: sw-bug
>Submitter-Id: current-users
>Arrival-Date: Fri Dec 10 04:40:22 GMT 2004
>Closed-Date:
>Last-Modified:
>Originator: Geoffrey Lassner
>Release: 5.3-RELEASE-p2
>Organization:
None
>Environment:
FreeBSD oak.zyni.com 5.3-RELEASE-p2 FreeBSD 5.3-RELEASE-p2 #2: Wed Dec 8 22:06:28 MST 2004 root at oak.zyni.com:/usr/src/sys/i386/compile/OAK-TEST i386
>Description:
Crashes with SMP enabled on 5.3 and 5.3-p2 (two systems identical hardware)
Fatal trap 12: page fault while in kernel mode
cpuid = 1; apic id = 00
fault virtual address = 0x4
fault code = supervisor write, page not present
instruction pointer = 0x8:0xc0464757
stack pointer = 0x10:0xd4233b10
frame pointer = 0x10:0xd4233b1c
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, def32 1, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 31 (swi1: net)
[thread 100027]
Stopped at ipllog+0x43: incl 0x4(%eax)
db> show thread
100115 (0xc1f2e7d0) fork_trampoline() at fork_trampoline
100100 (0xc1f2d000) sched_switch(c1f2d000,0,1) at sched_switch+0x143
100066 (0xc1a4e960) sched_switch(c1a4e960,0,2) at sched_switch+0x143
console 3%Fatal trap 12: page fault while in kernel mode
cpuid = 1; apic id = 00
fault virtual address = 0x4
fault code = supervisor write, page not present
instruction pointer = 0x8:0xc0464757
stack pointer = 0x10:0xd4233b10
frame pointer = 0x10:0xd4233b1c
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, def32 1, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 31 (swi1: net)
[thread 100027]
Stopped at ipllog+0x43: incl 0x4(%eax)
db> show thread
100115 (0xc1f2e7d0) fork_trampoline() at fork_trampoline
100100 (0xc1f2d000) sched_switch(c1f2d000,0,1) at sched_switch+0x143
100066 (0xc1a4e960) sched_switch(c1a4e960,0,2) at sched_switch+0x143
100051 (0xc1a4b190) sched_switch(c1a4b190,0,2) at sched_switch+0x143
100118 (0xc1f2ec80) sched_switch(c1f2ec80,0,1) at sched_switch+0x143
100102 (0xc1f2d320) sched_switch(c1f2d320,0,1) at sched_switch+0x143
100144 (0xc20d0640) sched_switch(c20d0640,0,1) at sched_switch+0x143
100104 (0xc1f2d640) sched_switch(c1f2d640,0,1) at sched_switch+0x143
100145 (0xc20d07d0) sched_switch(c20d07d0,0,1) at sched_switch+0x143
100121 (0xc1f53190) sched_switch(c1f53190,0,2) at sched_switch+0x143
100186 (0xc237e960) sched_switch(c237e960,0,1) at sched_switch+0x143
100138 (0xc20cfc80) sched_switch(c20cfc80,0,1) at sched_switch+0x143
100119 (0xc1f2ee10) sched_switch(c1f2ee10,0,2) at sched_switch+0x143
100135 (0xc20cf7d0) sched_switch(c20cf7d0,0,1) at sched_switch+0x143
100134 (0xc20cf640) sched_switch(c20cf640,0,1) at sched_switch+0x143
100058 (0xc1a4bc80) sched_switch(c1a4bc80,0,1) at sched_switch+0x143
100122 (0xc1f53320) sched_switch(c1f53320,0,1) at sched_switch+0x143
geoff at cyclone % more console
Fatal trap 12: page fault while in kernel mode
cpuid = 1; apic id = 00
fault virtual address = 0x4
fault code = supervisor write, page not present
instruction pointer = 0x8:0xc0464757
stack pointer = 0x10:0xd4233b10
frame pointer = 0x10:0xd4233b1c
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, def32 1, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 31 (swi1: net)
[thread 100027]
Stopped at ipllog+0x43: incl 0x4(%eax)
db> show thread
100115 (0xc1f2e7d0) fork_trampoline() at fork_trampoline
100100 (0xc1f2d000) sched_switch(c1f2d000,0,1) at sched_switch+0x143
100066 (0xc1a4e960) sched_switch(c1a4e960,0,2) at sched_switch+0x143
100051 (0xc1a4b190) sched_switch(c1a4b190,0,2) at sched_switch+0x143
100118 (0xc1f2ec80) sched_switch(c1f2ec80,0,1) at sched_switch+0x143
100102 (0xc1f2d320) sched_switch(c1f2d320,0,1) at sched_switch+0x143
100144 (0xc20d0640) sched_switch(c20d0640,0,1) at sched_switch+0x143
100104 (0xc1f2d640) sched_switch(c1f2d640,0,1) at sched_switch+0x143
100145 (0xc20d07d0) sched_switch(c20d07d0,0,1) at sched_switch+0x143
100121 (0xc1f53190) sched_switch(c1f53190,0,2) at sched_switch+0x143
100186 (0xc237e960) sched_switch(c237e960,0,1) at sched_switch+0x143
100138 (0xc20cfc80) sched_switch(c20cfc80,0,1) at sched_switch+0x143
100119 (0xc1f2ee10) sched_switch(c1f2ee10,0,2) at sched_switch+0x143
100135 (0xc20cf7d0) sched_switch(c20cf7d0,0,1) at sched_switch+0x143
100134 (0xc20cf640) sched_switch(c20cf640,0,1) at sched_switch+0x143
100058 (0xc1a4bc80) sched_switch(c1a4bc80,0,1) at sched_switch+0x143
100122 (0xc1f53320) sched_switch(c1f53320,0,1) at sched_switch+0x143
100081 (0xc1c6f190) sched_switch(c1c6f190,0,2) at sched_switch+0x143
100083 (0xc1c6f4b0) sched_switch(c1c6f4b0,0,1) at sched_switch+0x143
100069 (0xc1a4ee10) sched_switch(c1a4ee10,0,1) at sched_switch+0x143
100075 (0xc1c6e7d0) sched_switch(c1c6e7d0,0,1) at sched_switch+0x143
100113 (0xc1f2e4b0) sched_switch(c1f2e4b0,0,1) at sched_switch+0x143
100093 (0xc1c724b0) sched_switch(c1c724b0,0,1) at sched_switch+0x143
100084 (0xc1c6f640) sched_switch(c1c6f640,0,1) at sched_switch+0x143
100055 (0xc1a4b7d0) sched_switch(c1a4b7d0,0,1) at sched_switch+0x143
100062 (0xc1a4e320) sched_switch(c1a4e320,0,1) at sched_switch+0x143
100070 (0xc1c6e000) sched_switch(c1c6e000,0,1) at sched_switch+0x143
100111 (0xc1f2e190) sched_switch(c1f2e190,0,2) at sched_switch+0x143
100078 (0xc1c6ec80) sched_switch(c1c6ec80,0,1) at sched_switch+0x143
100110 (0xc1f2e000) sched_switch(c1f2e000,0,1) at sched_switch+0x143
100098 (0xc1c72c80) sched_switch(c1c72c80,0,1) at sched_switch+0x143
100099 (0xc1c72e10) sched_switch(c1c72e10,0,1) at sched_switch+0x143
100036 (0xc19a1960) sched_switch(c19a1960,0,1) at sched_switch+0x143
100085 (0xc1c6f7d0) sched_switch(c1c6f7d0,0,1) at sched_switch+0x143
100082 (0xc1c6f320) sched_switch(c1c6f320,0,1) at sched_switch+0x143
100112 (0xc1f2e320) sched_switch(c1f2e320,0,1) at sched_switch+0x143
100056 (0xc1a4b960) sched_switch(c1a4b960,0,1) at sched_switch+0x143
100109 (0xc1f2de10) sched_switch(c1f2de10,0,1) at sched_switch+0x143
100117 (0xc1f2eaf0) sched_switch(c1f2eaf0,0,1) at sched_switch+0x143
100092 (0xc1c72320) sched_switch(c1c72320,0,1) at sched_switch+0x143
100090 (0xc1c72000) sched_switch(c1c72000,0,1) at sched_switch+0x143
100080 (0xc1c6f000) sched_switch(c1c6f000,0,1) at sched_switch+0x143
100087 (0xc1c6faf0) sched_switch(c1c6faf0,0,1) at sched_switch+0x143
100065 (0xc1a4e7d0) sched_switch(c1a4e7d0,0,1) at sched_switch+0x143
100067 (0xc1a4eaf0) sched_switch(c1a4eaf0,0,1) at sched_switch+0x143
100101 (0xc1f2d190) sched_switch(c1f2d190,0,1) at sched_switch+0x143
100106 (0xc1f2d960) sched_switch(c1f2d960,0,1) at sched_switch+0x143
100126 (0xc1f53960) sched_switch(c1f53960,0,1) at sched_switch+0x143
100089 (0xc1c6fe10) sched_switch(c1c6fe10,0,1) at sched_switch+0x143
100096 (0xc1c72960) sched_switch(c1c72960,0,1) at sched_switch+0x143
100086 (0xc1c6f960) sched_switch(c1c6f960,0,1) at sched_switch+0x143
100039 (0xc19a1e10) sched_switch(c19a1e10,0,1) at sched_switch+0x143
100095 (0xc1c727d0) sched_switch(c1c727d0,0,1) at sched_switch+0x143
100057 (0xc1a4baf0) sched_switch(c1a4baf0,0,1) at sched_switch+0x143
100068 (0xc1a4ec80) sched_switch(c1a4ec80,0,1) at sched_switch+0x143
100091 (0xc1c72190) sched_switch(c1c72190,0,1) at sched_switch+0x143
100064 (0xc1a4e640) sched_switch(c1a4e640,0,1) at sched_switch+0x143
100063 (0xc1a4e4b0) sched_switch(c1a4e4b0,0,1) at sched_switch+0x143
100052 (0xc1a4b320) sched_switch(c1a4b320,0,1) at sched_switch+0x143
100094 (0xc1c72640) sched_switch(c1c72640,0,1) at sched_switch+0x143
100127 (0xc1f53af0) sched_switch(c1f53af0,0,2) at sched_switch+0x143
100128 (0xc1f53c80) sched_switch(c1f53c80,0,1) at sched_switch+0x143
100103 (0xc1f2d4b0) sched_switch(c1f2d4b0,0,1) at sched_switch+0x143
100097 (0xc1c72af0) sched_switch(c1c72af0,0,1) at sched_switch+0x143
100123 (0xc1f534b0) sched_switch(c1f534b0,0,1) at sched_switch+0x143
100108 (0xc1f2dc80) sched_switch(c1f2dc80,0,1) at sched_switch+0x143
100088 (0xc1c6fc80) sched_switch(c1c6fc80,0,1) at sched_switch+0x143
100059 (0xc1a4be10) sched_switch(c1a4be10,0,1) at sched_switch+0x143
100073 (0xc1c6e4b0) sched_switch(c1c6e4b0,0,1) at sched_switch+0x143
100072 (0xc1c6e320) sched_switch(c1c6e320,0,1) at sched_switch+0x143
100060 (0xc1a4e000) sched_switch(c1a4e000,0,1) at sched_switch+0x143
100038 (0xc19a1c80) sched_switch(c19a1c80,0,1) at sched_switch+0x143
100071 (0xc1c6e190) sched_switch(c1c6e190,0,1) at sched_switch+0x143
100053 (0xc1a4b4b0) sched_switch(c1a4b4b0,0,1) at sched_switch+0x143
100079 (0xc1c6ee10) sched_switch(c1c6ee10,0,1) at sched_switch+0x143
100074 (0xc1c6e640) sched_switch(c1c6e640,0,1) at sched_switch+0x143
100054 (0xc1a4b640) sched_switch(c1a4b640,0,1) at sched_switch+0x143
100076 (0xc1c6e960) sched_switch(c1c6e960,0,1) at sched_switch+0x143
100061 (0xc1a4e190) sched_switch(c1a4e190,0,1) at sched_switch+0x143
100077 (0xc1c6eaf0) sched_switch(c1c6eaf0,0,1) at sched_switch+0x143
100025 (0xc196e7d0) sched_switch(c196e7d0,0,1) at sched_switch+0x143
100024 (0xc196e640) sched_switch(c196e640,0,1) at sched_switch+0x143
100023 (0xc196e4b0) sched_switch(c196e4b0,0,1) at sched_switch+0x143
100049 (0xc1a48e10) sched_switch(c1a48e10,0,1) at sched_switch+0x143
100048 (0xc1a48c80) sched_switch(c1a48c80,0,1) at sched_switch+0x143
100047 (0xc1a48af0) sched_switch(c1a48af0,0,1) at sched_switch+0x143
100046 (0xc1a48960) sched_switch(c1a48960,0,1) at sched_switch+0x143
100045 (0xc1a487d0) sched_switch(c1a487d0,0,1) at sched_switch+0x143
100044 (0xc1a48640) sched_switch(c1a48640,0,1) at sched_switch+0x143
100043 (0xc1a484b0) sched_switch(c1a484b0,0,1) at sched_switch+0x143
100042 (0xc1a48320) sched_switch(c1a48320,0,1) at sched_switch+0x143
100041 (0xc1a48190) sched_switch(c1a48190,0,1) at sched_switch+0x143
100040 (0xc1a48000) fork_trampoline() at fork_trampoline
100012 (0xc196a320) fork_trampoline() at fork_trampoline
100011 (0xc196a190) sched_switch(c196a190,0,1) at sched_switch+0x143
100035 (0xc19a17d0) sched_switch(c19a17d0,0,1) at sched_switch+0x143
100034 (0xc19a1640) fork_trampoline() at fork_trampoline
100033 (0xc19a14b0) fork_trampoline() at fork_trampoline
100032 (0xc19a1320) sched_switch(c19a1320,0,1) at sched_switch+0x143
100031 (0xc19a1190) sched_switch(c19a1190,0,1) at sched_switch+0x143
100030 (0xc19a1000) sched_switch(18f1420,c1b0d800,c1c34800,c1bcab00,d5413c7c) at sched_switch+
0x143
100029 (0xc196ee10) sched_switch(c196ee10,0,1) at sched_switch+0x143
100028 (0xc196ec80) sched_switch(c196ec80,0,1) at sched_switch+0x143
100027 (0xc196eaf0) ipllog(0,d4233bd8,d4233b50,d4233b48,d4233b40) at ipllog+0x43
100026 (0xc196e960) fork_trampoline() at fork_trampoline
100000 (0xc1965000) sched_switch(c1965000,0,1) at sched_switch+0x143
100022 (0xc196e320) fork_trampoline() at fork_trampoline
100021 (0xc196e190) fork_trampoline() at fork_trampoline
100020 (0xc196e000) fork_trampoline() at fork_trampoline
100019 (0xc196ae10) fork_trampoline() at fork_trampoline
100018 (0xc196ac80) fork_trampoline() at fork_trampoline
100017 (0xc196aaf0) sched_switch(c196aaf0,0,1) at sched_switch+0x143
100016 (0xc196a960) sched_switch(c196a960,0,1) at sched_switch+0x143
100015 (0xc196a7d0) sched_switch(c196a7d0,0,1) at sched_switch+0x143
100014 (0xc196a640) fork_trampoline() at fork_trampoline
100013 (0xc196a4b0) sched_switch(c196a4b0,0,1) at sched_switch+0x143
100010 (0xc196a000) fork_trampoline() at fork_trampoline
100009 (0xc1965e10) fork_trampoline() at fork_trampoline
100008 (0xc1965c80) fork_trampoline() at fork_trampoline
100007 (0xc1965af0) fork_trampoline() at fork_trampoline
100006 (0xc1965960) fork_trampoline() at fork_trampoline
100005 (0xc19657d0) fork_trampoline() at fork_trampoline
100004 (0xc1965640) sched_switch(c1965640,0,1) at sched_switch+0x143
100003 (0xc19654b0) sched_switch(c19654b0,0,1) at sched_switch+0x143
100002 (0xc1965320) sched_switch(c1965320,0,1) at sched_switch+0x143
100001 (0xc1965190) sched_switch(c1965190,0,1) at sched_switch+0x143
0 (0xc08bce00) sched_switch(c08bce00,0,1) at sched_switch+0x143
db> show all procs
pid proc uarea uid ppid pgrp flag stat wmesg wchan cmd
28532 c20d5388 d6a8a000 0 753 753 0000002 [RUNQ] find
27240 c20d554c d6a8b000 0 735 27240 0004002 [SLPQ nanslp 0xc08c256c][SLP] sleep
27239 c20d4e20 d6a87000 0 735 27239 0004002 [RUNQ] tcpdump
23076 c20d4000 d6a7f000 1000 23075 23075 0004002 [RUNQ] ssh
23075 c20d4710 d6a83000 1000 718 23075 0004002 [SLPQ pipdwt 0xc2268480][SLP] scp
17334 c20d41c4 d6a80000 0 17331 17320 0000002 [SLPQ pause 0xc20d41fc][SLP] dump
17333 c20ce710 d6a6b000 0 17331 17320 0000002 [SLPQ pause 0xc20ce748][SLP] dump
17332 c20d454c d6a82000 0 17331 17320 0000002 [SLPQ pipdwt 0xc2268c00][SLP] dump
17331 c20ce54c d6a6a000 0 17320 17320 0000002 [SLPQ sbwait 0xc209ee84][SLP] dump
17321 c20d51c4 d6a89000 0 732 17320 0004002 [RUNQ] gzip
17320 c2378388 d6aff000 0 732 17320 0004002 [SLPQ wait 0xc2378388][SLP] dump
753 c20d11c4 d6a71000 0 724 753 0004003 [SLPQ wait 0xc20d11c4][SLP] find
752 c20d4c5c d6a86000 1000 700 752 0004002 [RUNQ] bzip2
743 c20d1710 d6a74000 0 740 738 0000002 [SLPQ pause 0xc20d1748][SLP] dump
742 c20d18d4 d6a75000 0 740 738 0000002 [SLPQ pause 0xc20d190c][SLP] dump
741 c1c5c388 d6896000 0 740 738 0000002 [SLPQ pipdwt 0xc1d9f180][SLP] dump
740 c2039000 d689e000 0 738 738 0000002 [SLPQ sbwait 0xc209eab8][SLP] dump
739 c1f29710 d693f000 0 729 738 0004002 [RUNQ] gzip
738 c1f29388 d693d000 0 729 738 0004002 [SLPQ wait 0xc1f29388][SLP] dump
735 c1c71388 d68e5000 0 712 735 0004003 [SLPQ pause 0xc1c713c0][SLP] tcsh
732 c1c6d8d4 d68df000 0 706 732 0004003 [SLPQ pause 0xc1c6d90c][SLP] tcsh
729 c20391c4 d6918000 0 694 729 0004003 [SLPQ pause 0xc20391fc][SLP] tcsh
724 c1f281c4 d6928000 1000 723 724 0004002 [SLPQ pause 0xc1f281fc][SLP] tcsh
723 c1f291c4 d693c000 1000 690 690 0000100 [SLPQ select 0xc08c7cc4][SLP] sshd
718 c1c5c8d4 d6899000 1000 717 718 0004002 [SLPQ pause 0xc1c5c90c][SLP] tcsh
717 c1a4ac5c d54a5000 1000 687 687 0000100 [SLPQ select 0xc08c7cc4][SLP] sshd
712 c1c711c4 d68e4000 1000 711 712 0004002 [SLPQ pause 0xc1c711fc][SLP] tcsh
711 c1f29a98 d697d000 1000 684 684 0000100 [RUNQ] sshd
706 c1c6d388 d68dc000 1000 705 706 0004002 [SLPQ pause 0xc1c6d3c0][SLP] tcsh
705 c1f29c5c d697e000 1000 681 681 0000100 [SLPQ select 0xc08c7cc4][SLP] sshd
700 c203954c d698d000 1000 699 700 0004002 [SLPQ pause 0xc2039584][SLP] tcsh
699 c2039710 d698f000 1000 678 678 0000100 [SLPQ select 0xc08c7cc4][SLP] sshd
694 c1a4a710 d5466000 1000 693 694 0004002 [SLPQ pause 0xc1a4a748][SLP] tcsh
693 c1f29000 d693b000 1000 675 675 0000100 [SLPQ select 0xc08c7cc4][SLP] sshd
690 c1f2954c d693e000 0 461 690 0000100 [SLPQ sbwait 0xc209b5a8][SLP] sshd
687 c1f298d4 d6940000 0 461 687 0000100 [SLPQ sbwait 0xc1f85e84][SLP] sshd
684 c1c5c710 d6898000 0 461 684 0000100 [SLPQ sbwait 0xc1f85bfc][SLP] sshd
681 c1f29e20 d697f000 0 461 681 0000100 [SLPQ sbwait 0xc1d95d40][SLP] sshd
678 c2039388 d692f000 0 461 678 0000100 [SLPQ sbwait 0xc1f86320][SLP] sshd
675 c1f28388 d6930000 0 461 675 0000100 [SLPQ sbwait 0xc1f86974][SLP] sshd
674 c1f28710 d6936000 0 617 617 0000000 [SLPQ accept 0xc1d9b03a][SLP] perl5.8.5
673 c1c6d000 d689d000 0 617 617 0000000 [SLPQ accept 0xc1d9b03a][SLP] perl5.8.5
geoff at cyclone %
geoff at cyclone %
geoff at cyclone %
geoff at cyclone %
geoff at cyclone %
geoff at cyclone %
geoff at cyclone %
geoff at cyclone %
geoff at cyclone % ls
console instruction_pointer mptable
geoff at cyclone % cat console
Fatal trap 12: page fault while in kernel mode
cpuid = 1; apic id = 00
fault virtual address = 0x4
fault code = supervisor write, page not present
instruction pointer = 0x8:0xc0464757
stack pointer = 0x10:0xd4233b10
frame pointer = 0x10:0xd4233b1c
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, def32 1, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 31 (swi1: net)
[thread 100027]
Stopped at ipllog+0x43: incl 0x4(%eax)
db> show thread
100115 (0xc1f2e7d0) fork_trampoline() at fork_trampoline
100100 (0xc1f2d000) sched_switch(c1f2d000,0,1) at sched_switch+0x143
100066 (0xc1a4e960) sched_switch(c1a4e960,0,2) at sched_switch+0x143
100051 (0xc1a4b190) sched_switch(c1a4b190,0,2) at sched_switch+0x143
100118 (0xc1f2ec80) sched_switch(c1f2ec80,0,1) at sched_switch+0x143
100102 (0xc1f2d320) sched_switch(c1f2d320,0,1) at sched_switch+0x143
100144 (0xc20d0640) sched_switch(c20d0640,0,1) at sched_switch+0x143
100104 (0xc1f2d640) sched_switch(c1f2d640,0,1) at sched_switch+0x143
100145 (0xc20d07d0) sched_switch(c20d07d0,0,1) at sched_switch+0x143
100121 (0xc1f53190) sched_switch(c1f53190,0,2) at sched_switch+0x143
100186 (0xc237e960) sched_switch(c237e960,0,1) at sched_switch+0x143
100138 (0xc20cfc80) sched_switch(c20cfc80,0,1) at sched_switch+0x143
100119 (0xc1f2ee10) sched_switch(c1f2ee10,0,2) at sched_switch+0x143
100135 (0xc20cf7d0) sched_switch(c20cf7d0,0,1) at sched_switch+0x143
100134 (0xc20cf640) sched_switch(c20cf640,0,1) at sched_switch+0x143
100058 (0xc1a4bc80) sched_switch(c1a4bc80,0,1) at sched_switch+0x143
100122 (0xc1f53320) sched_switch(c1f53320,0,1) at sched_switch+0x143
100081 (0xc1c6f190) sched_switch(c1c6f190,0,2) at sched_switch+0x143
100083 (0xc1c6f4b0) sched_switch(c1c6f4b0,0,1) at sched_switch+0x143
100069 (0xc1a4ee10) sched_switch(c1a4ee10,0,1) at sched_switch+0x143
100075 (0xc1c6e7d0) sched_switch(c1c6e7d0,0,1) at sched_switch+0x143
100113 (0xc1f2e4b0) sched_switch(c1f2e4b0,0,1) at sched_switch+0x143
100093 (0xc1c724b0) sched_switch(c1c724b0,0,1) at sched_switch+0x143
100084 (0xc1c6f640) sched_switch(c1c6f640,0,1) at sched_switch+0x143
100055 (0xc1a4b7d0) sched_switch(c1a4b7d0,0,1) at sched_switch+0x143
100062 (0xc1a4e320) sched_switch(c1a4e320,0,1) at sched_switch+0x143
100070 (0xc1c6e000) sched_switch(c1c6e000,0,1) at sched_switch+0x143
100111 (0xc1f2e190) sched_switch(c1f2e190,0,2) at sched_switch+0x143
100078 (0xc1c6ec80) sched_switch(c1c6ec80,0,1) at sched_switch+0x143
100110 (0xc1f2e000) sched_switch(c1f2e000,0,1) at sched_switch+0x143
100098 (0xc1c72c80) sched_switch(c1c72c80,0,1) at sched_switch+0x143
100099 (0xc1c72e10) sched_switch(c1c72e10,0,1) at sched_switch+0x143
100036 (0xc19a1960) sched_switch(c19a1960,0,1) at sched_switch+0x143
100085 (0xc1c6f7d0) sched_switch(c1c6f7d0,0,1) at sched_switch+0x143
100082 (0xc1c6f320) sched_switch(c1c6f320,0,1) at sched_switch+0x143
100112 (0xc1f2e320) sched_switch(c1f2e320,0,1) at sched_switch+0x143
100056 (0xc1a4b960) sched_switch(c1a4b960,0,1) at sched_switch+0x143
100109 (0xc1f2de10) sched_switch(c1f2de10,0,1) at sched_switch+0x143
100117 (0xc1f2eaf0) sched_switch(c1f2eaf0,0,1) at sched_switch+0x143
100092 (0xc1c72320) sched_switch(c1c72320,0,1) at sched_switch+0x143
100090 (0xc1c72000) sched_switch(c1c72000,0,1) at sched_switch+0x143
100080 (0xc1c6f000) sched_switch(c1c6f000,0,1) at sched_switch+0x143
100087 (0xc1c6faf0) sched_switch(c1c6faf0,0,1) at sched_switch+0x143
100065 (0xc1a4e7d0) sched_switch(c1a4e7d0,0,1) at sched_switch+0x143
100067 (0xc1a4eaf0) sched_switch(c1a4eaf0,0,1) at sched_switch+0x143
100101 (0xc1f2d190) sched_switch(c1f2d190,0,1) at sched_switch+0x143
100106 (0xc1f2d960) sched_switch(c1f2d960,0,1) at sched_switch+0x143
100126 (0xc1f53960) sched_switch(c1f53960,0,1) at sched_switch+0x143
100089 (0xc1c6fe10) sched_switch(c1c6fe10,0,1) at sched_switch+0x143
100096 (0xc1c72960) sched_switch(c1c72960,0,1) at sched_switch+0x143
100086 (0xc1c6f960) sched_switch(c1c6f960,0,1) at sched_switch+0x143
100039 (0xc19a1e10) sched_switch(c19a1e10,0,1) at sched_switch+0x143
100095 (0xc1c727d0) sched_switch(c1c727d0,0,1) at sched_switch+0x143
100057 (0xc1a4baf0) sched_switch(c1a4baf0,0,1) at sched_switch+0x143
100068 (0xc1a4ec80) sched_switch(c1a4ec80,0,1) at sched_switch+0x143
100091 (0xc1c72190) sched_switch(c1c72190,0,1) at sched_switch+0x143
100064 (0xc1a4e640) sched_switch(c1a4e640,0,1) at sched_switch+0x143
100063 (0xc1a4e4b0) sched_switch(c1a4e4b0,0,1) at sched_switch+0x143
100052 (0xc1a4b320) sched_switch(c1a4b320,0,1) at sched_switch+0x143
100094 (0xc1c72640) sched_switch(c1c72640,0,1) at sched_switch+0x143
100127 (0xc1f53af0) sched_switch(c1f53af0,0,2) at sched_switch+0x143
100128 (0xc1f53c80) sched_switch(c1f53c80,0,1) at sched_switch+0x143
100103 (0xc1f2d4b0) sched_switch(c1f2d4b0,0,1) at sched_switch+0x143
100097 (0xc1c72af0) sched_switch(c1c72af0,0,1) at sched_switch+0x143
100123 (0xc1f534b0) sched_switch(c1f534b0,0,1) at sched_switch+0x143
100108 (0xc1f2dc80) sched_switch(c1f2dc80,0,1) at sched_switch+0x143
100088 (0xc1c6fc80) sched_switch(c1c6fc80,0,1) at sched_switch+0x143
100059 (0xc1a4be10) sched_switch(c1a4be10,0,1) at sched_switch+0x143
100073 (0xc1c6e4b0) sched_switch(c1c6e4b0,0,1) at sched_switch+0x143
100072 (0xc1c6e320) sched_switch(c1c6e320,0,1) at sched_switch+0x143
100060 (0xc1a4e000) sched_switch(c1a4e000,0,1) at sched_switch+0x143
100038 (0xc19a1c80) sched_switch(c19a1c80,0,1) at sched_switch+0x143
100071 (0xc1c6e190) sched_switch(c1c6e190,0,1) at sched_switch+0x143
100053 (0xc1a4b4b0) sched_switch(c1a4b4b0,0,1) at sched_switch+0x143
100079 (0xc1c6ee10) sched_switch(c1c6ee10,0,1) at sched_switch+0x143
100074 (0xc1c6e640) sched_switch(c1c6e640,0,1) at sched_switch+0x143
100054 (0xc1a4b640) sched_switch(c1a4b640,0,1) at sched_switch+0x143
100076 (0xc1c6e960) sched_switch(c1c6e960,0,1) at sched_switch+0x143
100061 (0xc1a4e190) sched_switch(c1a4e190,0,1) at sched_switch+0x143
100077 (0xc1c6eaf0) sched_switch(c1c6eaf0,0,1) at sched_switch+0x143
100025 (0xc196e7d0) sched_switch(c196e7d0,0,1) at sched_switch+0x143
100024 (0xc196e640) sched_switch(c196e640,0,1) at sched_switch+0x143
100023 (0xc196e4b0) sched_switch(c196e4b0,0,1) at sched_switch+0x143
100049 (0xc1a48e10) sched_switch(c1a48e10,0,1) at sched_switch+0x143
100048 (0xc1a48c80) sched_switch(c1a48c80,0,1) at sched_switch+0x143
100047 (0xc1a48af0) sched_switch(c1a48af0,0,1) at sched_switch+0x143
100046 (0xc1a48960) sched_switch(c1a48960,0,1) at sched_switch+0x143
100045 (0xc1a487d0) sched_switch(c1a487d0,0,1) at sched_switch+0x143
100044 (0xc1a48640) sched_switch(c1a48640,0,1) at sched_switch+0x143
100043 (0xc1a484b0) sched_switch(c1a484b0,0,1) at sched_switch+0x143
100042 (0xc1a48320) sched_switch(c1a48320,0,1) at sched_switch+0x143
100041 (0xc1a48190) sched_switch(c1a48190,0,1) at sched_switch+0x143
100040 (0xc1a48000) fork_trampoline() at fork_trampoline
100012 (0xc196a320) fork_trampoline() at fork_trampoline
100011 (0xc196a190) sched_switch(c196a190,0,1) at sched_switch+0x143
100035 (0xc19a17d0) sched_switch(c19a17d0,0,1) at sched_switch+0x143
100034 (0xc19a1640) fork_trampoline() at fork_trampoline
100033 (0xc19a14b0) fork_trampoline() at fork_trampoline
100032 (0xc19a1320) sched_switch(c19a1320,0,1) at sched_switch+0x143
100031 (0xc19a1190) sched_switch(c19a1190,0,1) at sched_switch+0x143
100030 (0xc19a1000) sched_switch(18f1420,c1b0d800,c1c34800,c1bcab00,d5413c7c) at sched_switch+0x143
100029 (0xc196ee10) sched_switch(c196ee10,0,1) at sched_switch+0x143
100028 (0xc196ec80) sched_switch(c196ec80,0,1) at sched_switch+0x143
100027 (0xc196eaf0) ipllog(0,d4233bd8,d4233b50,d4233b48,d4233b40) at ipllog+0x43
100026 (0xc196e960) fork_trampoline() at fork_trampoline
100000 (0xc1965000) sched_switch(c1965000,0,1) at sched_switch+0x143
100022 (0xc196e320) fork_trampoline() at fork_trampoline
100021 (0xc196e190) fork_trampoline() at fork_trampoline
100020 (0xc196e000) fork_trampoline() at fork_trampoline
100019 (0xc196ae10) fork_trampoline() at fork_trampoline
100018 (0xc196ac80) fork_trampoline() at fork_trampoline
100017 (0xc196aaf0) sched_switch(c196aaf0,0,1) at sched_switch+0x143
100016 (0xc196a960) sched_switch(c196a960,0,1) at sched_switch+0x143
100015 (0xc196a7d0) sched_switch(c196a7d0,0,1) at sched_switch+0x143
100014 (0xc196a640) fork_trampoline() at fork_trampoline
100013 (0xc196a4b0) sched_switch(c196a4b0,0,1) at sched_switch+0x143
100010 (0xc196a000) fork_trampoline() at fork_trampoline
100009 (0xc1965e10) fork_trampoline() at fork_trampoline
100008 (0xc1965c80) fork_trampoline() at fork_trampoline
100007 (0xc1965af0) fork_trampoline() at fork_trampoline
100006 (0xc1965960) fork_trampoline() at fork_trampoline
100005 (0xc19657d0) fork_trampoline() at fork_trampoline
100004 (0xc1965640) sched_switch(c1965640,0,1) at sched_switch+0x143
100003 (0xc19654b0) sched_switch(c19654b0,0,1) at sched_switch+0x143
100002 (0xc1965320) sched_switch(c1965320,0,1) at sched_switch+0x143
100001 (0xc1965190) sched_switch(c1965190,0,1) at sched_switch+0x143
0 (0xc08bce00) sched_switch(c08bce00,0,1) at sched_switch+0x143
db> show all procs
pid proc uarea uid ppid pgrp flag stat wmesg wchan cmd
28532 c20d5388 d6a8a000 0 753 753 0000002 [RUNQ] find
27240 c20d554c d6a8b000 0 735 27240 0004002 [SLPQ nanslp 0xc08c256c][SLP] sleep
27239 c20d4e20 d6a87000 0 735 27239 0004002 [RUNQ] tcpdump
23076 c20d4000 d6a7f000 1000 23075 23075 0004002 [RUNQ] ssh
23075 c20d4710 d6a83000 1000 718 23075 0004002 [SLPQ pipdwt 0xc2268480][SLP] scp
17334 c20d41c4 d6a80000 0 17331 17320 0000002 [SLPQ pause 0xc20d41fc][SLP] dump
17333 c20ce710 d6a6b000 0 17331 17320 0000002 [SLPQ pause 0xc20ce748][SLP] dump
17332 c20d454c d6a82000 0 17331 17320 0000002 [SLPQ pipdwt 0xc2268c00][SLP] dump
17331 c20ce54c d6a6a000 0 17320 17320 0000002 [SLPQ sbwait 0xc209ee84][SLP] dump
17321 c20d51c4 d6a89000 0 732 17320 0004002 [RUNQ] gzip
17320 c2378388 d6aff000 0 732 17320 0004002 [SLPQ wait 0xc2378388][SLP] dump
753 c20d11c4 d6a71000 0 724 753 0004003 [SLPQ wait 0xc20d11c4][SLP] find
752 c20d4c5c d6a86000 1000 700 752 0004002 [RUNQ] bzip2
743 c20d1710 d6a74000 0 740 738 0000002 [SLPQ pause 0xc20d1748][SLP] dump
742 c20d18d4 d6a75000 0 740 738 0000002 [SLPQ pause 0xc20d190c][SLP] dump
741 c1c5c388 d6896000 0 740 738 0000002 [SLPQ pipdwt 0xc1d9f180][SLP] dump
740 c2039000 d689e000 0 738 738 0000002 [SLPQ sbwait 0xc209eab8][SLP] dump
739 c1f29710 d693f000 0 729 738 0004002 [RUNQ] gzip
738 c1f29388 d693d000 0 729 738 0004002 [SLPQ wait 0xc1f29388][SLP] dump
735 c1c71388 d68e5000 0 712 735 0004003 [SLPQ pause 0xc1c713c0][SLP] tcsh
732 c1c6d8d4 d68df000 0 706 732 0004003 [SLPQ pause 0xc1c6d90c][SLP] tcsh
729 c20391c4 d6918000 0 694 729 0004003 [SLPQ pause 0xc20391fc][SLP] tcsh
724 c1f281c4 d6928000 1000 723 724 0004002 [SLPQ pause 0xc1f281fc][SLP] tcsh
723 c1f291c4 d693c000 1000 690 690 0000100 [SLPQ select 0xc08c7cc4][SLP] sshd
718 c1c5c8d4 d6899000 1000 717 718 0004002 [SLPQ pause 0xc1c5c90c][SLP] tcsh
717 c1a4ac5c d54a5000 1000 687 687 0000100 [SLPQ select 0xc08c7cc4][SLP] sshd
712 c1c711c4 d68e4000 1000 711 712 0004002 [SLPQ pause 0xc1c711fc][SLP] tcsh
711 c1f29a98 d697d000 1000 684 684 0000100 [RUNQ] sshd
706 c1c6d388 d68dc000 1000 705 706 0004002 [SLPQ pause 0xc1c6d3c0][SLP] tcsh
705 c1f29c5c d697e000 1000 681 681 0000100 [SLPQ select 0xc08c7cc4][SLP] sshd
700 c203954c d698d000 1000 699 700 0004002 [SLPQ pause 0xc2039584][SLP] tcsh
699 c2039710 d698f000 1000 678 678 0000100 [SLPQ select 0xc08c7cc4][SLP] sshd
694 c1a4a710 d5466000 1000 693 694 0004002 [SLPQ pause 0xc1a4a748][SLP] tcsh
693 c1f29000 d693b000 1000 675 675 0000100 [SLPQ select 0xc08c7cc4][SLP] sshd
690 c1f2954c d693e000 0 461 690 0000100 [SLPQ sbwait 0xc209b5a8][SLP] sshd
687 c1f298d4 d6940000 0 461 687 0000100 [SLPQ sbwait 0xc1f85e84][SLP] sshd
684 c1c5c710 d6898000 0 461 684 0000100 [SLPQ sbwait 0xc1f85bfc][SLP] sshd
681 c1f29e20 d697f000 0 461 681 0000100 [SLPQ sbwait 0xc1d95d40][SLP] sshd
678 c2039388 d692f000 0 461 678 0000100 [SLPQ sbwait 0xc1f86320][SLP] sshd
675 c1f28388 d6930000 0 461 675 0000100 [SLPQ sbwait 0xc1f86974][SLP] sshd
674 c1f28710 d6936000 0 617 617 0000000 [SLPQ accept 0xc1d9b03a][SLP] perl5.8.5
673 c1c6d000 d689d000 0 617 617 0000000 [SLPQ accept 0xc1d9b03a][SLP] perl5.8.5
672 c1f28c5c d6939000 0 617 617 0000000 [SLPQ accept 0xc1d9b03a][SLP] perl5.8.5
671 c1c71a98 d68e9000 0 617 617 0000000 [SLPQ accept 0xc1d9b03a][SLP] perl5.8.5
670 c1c71710 d68e7000 0 617 617 0000000 [SLPQ accept 0xc1d9b03a][SLP] perl5.8.5
669 c20398d4 d69c2000 0 1 669 0004002 [SLPQ ttyin 0xc1c1d410][SLP] getty
668 c2039a98 d69c3000 0 1 668 0004002 [SLPQ ttyin 0xc1afa210][SLP] getty
667 c2039c5c d69c4000 0 1 667 0004002 [SLPQ ttyin 0xc1afa410][SLP] getty
666 c1f288d4 d6937000 0 1 666 0004002 [SLPQ ttyin 0xc1afa610][SLP] getty
665 c1c71c5c d68ea000 0 1 665 0004002 [SLPQ ttyin 0xc1afa810][SLP] getty
664 c1f28e20 d693a000 0 1 664 0004002 [SLPQ ttyin 0xc1afaa10][SLP] getty
663 c1a4a1c4 d5463000 0 1 663 0004002 [SLPQ ttyin 0xc1a62210][SLP] getty
662 c1c71e20 d6909000 0 1 662 0004002 [SLPQ ttyin 0xc195b610][SLP] getty
661 c1c5c54c d6897000 0 1 661 0004002 [SLPQ ttyin 0xc195b410][SLP] getty
656 c1c7154c d68e6000 0 1 656 0000000 [SLPQ select 0xc08c7cc4][SLP] inetd
625 c1f2854c d6935000 0 1 624 0000000 [SLPQ select 0xc08c7cc4][SLP] snmpd
617 c1a4a8d4 d5467000 0 1 617 0000000 [SLPQ pause 0xc1a4a90c][SLP] perl5.8.5
615 c1a4aa98 d54a4000 20001 560 560 0000100 [SLPQ lockf 0xc1f252c0][SLP] perl
614 c1c5ce20 d689c000 20001 560 560 0000100 [SLPQ accept 0xc1ceb17e][SLP] perl
605 c1f28000 d6919000 0 1 605 0000100 [SLPQ select 0xc08c7cc4][SLP] perl
604 c1c718d4 d68e8000 88 579 47 000c182 (threaded) mysqld
thread 0xc1f53af0 ksegrp 0xc1a49b60 [RUNQ]
thread 0xc1f53c80 ksegrp 0xc196c460 [SLPQ kserel 0xc196c4a0][SLP]
thread 0xc1f2d4b0 ksegrp 0xc1a49b60 [SLPQ kserel 0xc1a49ba0][SLP]
thread 0xc1c72af0 ksegrp 0xc1a49b60 [SLPQ select 0xc08c7cc4][SLP]
thread 0xc1f534b0 ksegrp 0xc1a49150 [SLPQ sigwait 0xd699ec2c][SLP]
thread 0xc1f2dc80 ksegrp 0xc1a491c0 [SLPQ ksesigwait 0xc1c719d4][SLP]
579 c1f28a98 d6938000 0 1 47 0004002 [SLPQ wait 0xc1f28a98][SLP] sh
567 c1c5c1c4 d6895000 106 1 567 0000100 [SLPQ accept 0xc1cebce2][SLP] clamd
560 c1c6dc5c d68e1000 20001 1 560 0000100 [SLPQ select 0xc08c7cc4][SLP] perl
474 c1c6de20 d68e2000 0 1 474 0000000 [SLPQ nanslp 0xc08c256c][SLP] cron
461 c1c5c000 d6893000 0 1 461 0000100 [SLPQ select 0xc08c7cc4][SLP] sshd
434 c1a4a388 d5464000 0 1 47 0004002 [SLPQ nanslp 0xc08c256c][SLP] perl
433 c1c71000 d68e3000 20000 431 431 0004100 [SLPQ select 0xc08c7cc4][SLP] qmgr
432 c1c5cc5c d689b000 20000 431 431 0004100 [SLPQ select 0xc08c7cc4][SLP] pickup
431 c1c6d1c4 d68db000 0 1 431 0004100 [SLPQ select 0xc08c7cc4][SLP] master
384 c1c6da98 d68e0000 65534 1 384 0000100 [SLPQ bpf 0xc1ce6400][SLP] snort
290 c1c5ca98 d689a000 0 1 290 0000000 [SLPQ ffsfsn 0xc1cfb134][SLP] syslogd
270 c1c6d710 d68de000 0 1 270 0000000 [SLPQ select 0xc08c7cc4][SLP] devd
187 c1a4ae20 d54a6000 0 1 187 0000000 [SLPQ pause 0xc1a4ae58][SLP] adjkerntz
165 c1c6d54c d68dd000 0 1 165 0000000 [SLPQ nanslp 0xc08c256c][SLP] ipmon
46 c19b0a98 d5438000 0 0 0 0000204 [SLPQ - 0xd422dd18][SLP] schedcpu
45 c19b0c5c d5439000 0 0 0 0000204 [SLPQ hptrdy 0xc0768be4][SLP] hpt_wt
44 c19b0e20 d543a000 0 0 0 0000204 [SLPQ vlruwt 0xc19b0e20][SLP] vnlru
43 c1a47000 d543b000 0 0 0 0000204 [SLPQ syncer 0xc08c22ec][SLP] syncer
42 c1a471c4 d545a000 0 0 0 0000204 [SLPQ psleep 0xc08c828c][SLP] bufdaemon
41 c1a47388 d545b000 0 0 0 000020c [RUNQ] pagezero
40 c1a4754c d545c000 0 0 0 0000204 [SLPQ psleep 0xc08d0024][SLP] vmdaemon
39 c1a47710 d545d000 0 0 0 0000204 [SLPQ psleep 0xc08cffe0][SLP] pagedaemon
38 c1a478d4 d545e000 0 0 0 0000204 [IWAIT] swi0: sio
9 c1a47a98 d545f000 0 0 0 0000204 [SLPQ - 0xc19a8d3c][SLP] fdc0
8 c1a47c5c d5460000 0 0 0 0000204 [SLPQ idle 0xc195a400][SLP] aic_recovery0
7 c1a47e20 d5461000 0 0 0 0000204 [SLPQ idle 0xc195a400][SLP] aic_recovery0
37 c1a4a000 d5462000 0 0 0 0000204 [IWAIT] swi6:+
36 c19a054c d540e000 0 0 0 0000204 [IWAIT] swi6: task queue
6 c19a0710 d540f000 0 0 0 0000204 [SLPQ - 0xc19ec580][SLP] kqueue taskq
35 c19a08d4 d5410000 0 0 0 0000204 [IWAIT] swi3: cambio
34 c19a0a98 d542f000 0 0 0 0000204 [IWAIT] swi2: camnet
33 c19a0c5c d5430000 0 0 0 0000204 [IWAIT] swi6:+
5 c19a0e20 d5431000 0 0 0 0000204 [SLPQ - 0xc19ec7c0][SLP] thread taskq
32 c19b0000 d5432000 0 0 0 0000204 [SLPQ - 0xc08b8100][SLP] yarrow
4 c19b01c4 d5433000 0 0 0 0000204 [CPU 0] g_down
3 c19b0388 d5434000 0 0 0 0000204 [SLPQ - 0xc08bcb24][SLP] g_up
2 c19b054c d5435000 0 0 0 0000204 [SLPQ - 0xc08bcb1c][SLP] g_event
31 c19b0710 d5436000 0 0 0 0000204 [CPU 1] swi1: net
30 c19b08d4 d5437000 0 0 0 0000204 [IWAIT] swi4: vm
29 c196d1c4 d421a000 0 0 0 000020c [RUNQ] swi5: clock sio
28 c196d388 d421b000 0 0 0 0000204 [IWAIT] irq0: clk
27 c196d54c d423a000 0 0 0 0000204 [IWAIT] irq15:
26 c196d710 d423b000 0 0 0 0000204 [IWAIT] irq14: atapci0 ata0
25 c196d8d4 d423c000 0 0 0 0000204 [IWAIT] irq13:
24 c196da98 d423d000 0 0 0 0000204 [IWAIT] irq12:
23 c196dc5c d423e000 0 0 0 0000204 [IWAIT] irq11: fxp1
22 c196de20 d423f000 0 0 0 0000204 [IWAIT] irq10: xl0 fxp0
21 c19a0000 d540b000 0 0 0 0000204 [IWAIT] irq9: ahc0
20 c19a01c4 d540c000 0 0 0 0000204 [IWAIT] irq8: rtc
19 c19a0388 d540d000 0 0 0 0000204 [IWAIT] irq7: ep0
18 c1964000 d41d4000 0 0 0 0000204 [IWAIT] irq6: fdc0
17 c19641c4 d4211000 0 0 0 0000204 [IWAIT] irq5:
16 c1964388 d4212000 0 0 0 0000204 [IWAIT] irq4: sio0
15 c196454c d4213000 0 0 0 0000204 [IWAIT] irq3: sio1
14 c1964710 d4214000 0 0 0 0000204 [IWAIT] irq2:
13 c19648d4 d4215000 0 0 0 0000204 [IWAIT] irq1: atkbd0
12 c1964a98 d4216000 0 0 0 000020c [Can run] idle: cpu0
11 c1964c5c d4217000 0 0 0 000020c [Can run] idle: cpu1
1 c1964e20 d4218000 0 0 1 0004200 [SLPQ wait 0xc1964e20][SLP] init
10 c196d000 d4219000 0 0 0 0000204 [SLPQ ktrace 0xc08c0518][SLP] ktrace
0 c08bcc20 c0c1f000 0 0 0 0000200 [SLPQ sched 0xc08bcc20][SLP] swapper
db> show pcpu 0
cpuid = 0
curthread = 0xc19a1000: pid 4 "g_down"
curpcb = 0xd5413da0
fpcurthread = none
idlethread = 0xc1965640: pid 12 "idle: cpu0"
APIC ID = 1
currentldt = 0x28
db> show pcpu 1
cpuid = 1
curthread = 0xc196eaf0: pid 31 "swi1: net"
curpcb = 0xd4233da0
fpcurthread = none
idlethread = 0xc19654b0: pid 11 "idle: cpu1"
APIC ID = 0
currentldt = 0x28
db> trace
ipllog(0,d4233bd8,d4233b50,d4233b48,d4233b40) at ipllog+0x43
ipflog(1a,c2692820,d4233bd8,c1a72500) at ipflog+0x18f
fr_check(c2692820,14,c1a44000,0,d4233c80) at fr_check+0x820
fr_check_wrapper(0,d4233c80,c1a44000,1,0) at fr_check_wrapper+0x2a
pfil_run_hooks(c08c99c0,d4233ccc,c1a44000,1,0) at pfil_run_hooks+0xe3
ip_input(c1a72500) at ip_input+0x211
netisr_processqueue(c08c8c58) at netisr_processqueue+0xa3
swi_net(0) at swi_net+0xf2
ithread_loop(c1962680,d4233d48) at ithread_loop+0x159
fork_exit(c060bb5c,c1962680,d4233d48) at fork_exit+0x75
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xd4233d7c, ebp = 0 ---
db> call doadump
Dumping 512 MB
16 32 48 64 80 96 112 128 144 160 176 192 208 224 240 256 272 288 304 320 336 352 368 384 400 416 432 448 464 480 496
Dump complete
0xf
db> reset
cpu_reset called on cpu#1
cpu_reset: Restarting BSP
cpu_reset_proxy: Stopped CPU 1
oak# nm -n /boot/kernel/kernel | grep c04647
c0464714 T ipllog
oak#
oak# mptable -dmesg -verbose -grope
===============================================================================
MPTable, version 2.0.15
looking for EBDA pointer @ 0x040e, found, searching EBDA @ 0x0009f800
searching CMOS 'top of mem' @ 0x0009f400 (637K)
searching default 'top of mem' @ 0x0009fc00 (639K)
searching BIOS @ 0x000f0000
MP FPS found in BIOS @ physical addr: 0x000f99a0
-------------------------------------------------------------------------------
MP Floating Pointer Structure:
location: BIOS
physical address: 0x000f99a0
signature: '_MP_'
length: 16 bytes
version: 1.1
checksum: 0x4b
mode: Virtual Wire
-------------------------------------------------------------------------------
MP Config Table Header:
physical address: 0x000f99b0
signature: 'PCMP'
base table length: 236
version: 1.1
checksum: 0xe0
OEM ID: 'HP '
Product ID: 'XX '
OEM table pointer: 0x00000000
OEM table size: 0
entry count: 21
local APIC address: 0xfee00000
extended table length: 0
extended table checksum: 0
-------------------------------------------------------------------------------
MP Config Base Table Entries:
--
Processors: APIC ID Version State Family Model Step Flags
1 0x11 BSP, usable 6 1 0 0xfbff
0 0x11 AP, usable 6 1 0 0xfbff
--
Bus: Bus ID Type
1 ISA
0 PCI
--
I/O APICs: APIC ID Version State Address
16 0x11 usable 0xfec00000
--
I/O Ints: Type Polarity Trigger Bus ID IRQ APIC ID PIN#
INT conforms conforms 1 1 255 1
INT conforms conforms 1 0 255 2
INT conforms conforms 1 3 255 3
INT conforms conforms 1 4 255 4
INT conforms conforms 1 5 255 5
INT conforms conforms 1 6 255 6
INT conforms conforms 1 7 255 7
INT conforms conforms 1 8 255 8
INT conforms conforms 1 9 255 9
INT conforms conforms 1 10 255 10
INT conforms conforms 1 11 255 11
INT conforms conforms 1 12 255 12
INT conforms conforms 1 13 255 13
INT conforms conforms 1 14 255 14
INT conforms conforms 1 15 255 15
--
Local Ints: Type Polarity Trigger Bus ID IRQ APIC ID PIN#
NMI conforms conforms 1 2 255 1
-------------------------------------------------------------------------------
dmesg output:
Copyright (c) 1992-2004 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD 5.3-RELEASE-p2 #2: Wed Dec 8 22:06:28 MST 2004
root at oak.zyni.com:/usr/src/sys/i386/compile/OAK-TEST
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Pentium Pro (199.74-MHz 686-class CPU)
Origin = "GenuineIntel" Id = 0x619 Stepping = 9
Features=0xfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV>
real memory = 536870912 (512 MB)
avail memory = 515813376 (491 MB)
MPTable: <HP XX >
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
cpu0 (BSP): APIC ID: 1
cpu1 (AP): APIC ID: 0
ioapic0: Changing APIC ID to 16
ioapic0: Assuming intbase of 0
MPTable: Ignoring global interrupt entry for pin 1
MPTable: Ignoring global interrupt entry for pin 2
MPTable: Ignoring global interrupt entry for pin 3
MPTable: Ignoring global interrupt entry for pin 4
MPTable: Ignoring global interrupt entry for pin 5
MPTable: Ignoring global interrupt entry for pin 6
MPTable: Ignoring global interrupt entry for pin 7
MPTable: Ignoring global interrupt entry for pin 8
MPTable: Ignoring global interrupt entry for pin 9
MPTable: Ignoring global interrupt entry for pin 10
MPTable: Ignoring global interrupt entry for pin 11
MPTable: Ignoring global interrupt entry for pin 12
MPTable: Ignoring global interrupt entry for pin 13
MPTable: Ignoring global interrupt entry for pin 14
MPTable: Ignoring global interrupt entry for pin 15
ioapic0 <Version 1.1> irqs 0-15 on motherboard
npx0: [FAST]
npx0: <math processor> on motherboard
npx0: INT 16 interface
pcib0: <Intel 82454KX/GX (Orion) host to PCI bridge> pcibus 0 on motherboard
pci0: <PCI bus> on pcib0
isab0: <PCI-ISA bridge> at device 0.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <CMD 646 WDMA2 controller> port 0xfc00-0xfc0f,0x376,0x170-0x177,0x3f6,0x1f0-0x1f7 irq 14 at device 1.0 on pci0
atapci0: HW has secondary channel disabled
ata0: channel #0 on atapci0
ahc0: <Adaptec aic7880 Ultra SCSI adapter> port 0xf800-0xf8ff mem 0xffbfc000-0xffbfcfff irq 9 at device 2.0 on pci0
ahc0: Using left over BIOS settings
ahc0: [GIANT-LOCKED]
aic7880: Ultra Single Channel A, SCSI Id=7, 16/253 SCBs
xl0: <3Com 3c905B-TX Fast Etherlink XL> port 0xf400-0xf47f mem 0xffbec000-0xffbec07f irq 10 at device 11.0 on pci0
miibus0: <MII bus> on xl0
xlphy0: <3Com internal media interface> on miibus0
xlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
xl0: Ethernet address: 00:10:5a:0d:79:54
fxp0: <Intel 82557 Pro/100 Ethernet> port 0xf000-0xf01f mem 0xffa00000-0xffafffff,0xffbdc000-0xffbdcfff irq 10 at device 12.0 on pci0
miibus1: <MII bus> on fxp0
nsphy0: <DP83840 10/100 media interface> on miibus1
nsphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
fxp0: Ethernet address: 00:a0:c9:57:e8:69
fxp1: <Intel 82557 Pro/100 Ethernet> port 0xec00-0xec1f mem 0xff900000-0xff9fffff,0xffbcc000-0xffbccfff irq 11 at device 13.0 on pci0
miibus2: <MII bus> on fxp1
nsphy1: <DP83840 10/100 media interface> on miibus2
nsphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
fxp1: Ethernet address: 00:a0:c9:3e:19:1d
pci0: <display, VGA> at device 14.0 (no driver attached)
pci0: <memory, RAM> at device 20.0 (no driver attached)
cpu0 on motherboard
cpu1 on motherboard
orm0: <ISA Option ROMs> at iomem 0xc8000-0xcc7ff,0xc0000-0xc7fff on isa0
pmtimer0 on isa0
ep0: <3Com 3C509-TP EtherLink III> at port 0x290-0x29f irq 7 on isa0
ep0: Ethernet address: 00:60:8c:b9:5e:18
atkbdc0: <Keyboard controller (i8042)> at port 0x64,0x60 on isa0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
fdc0: <Enhanced floppy controller> at port 0x3f0-0x3f5 irq 6 drq 2 on isa0
fdc0: [FAST]
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
ppc0: parallel port not found.
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x100>
sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0
sio0: type 16550A, console
sio1 at port 0x2f8-0x2ff irq 3 on isa0
sio1: type 16550A
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
unknown: <PNP0303> can't assign resources (port)
unknown: <PNP0501> can't assign resources (port)
unknown: <PNP0501> can't assign resources (port)
ppc1: parallel port not found.
unknown: <PNP0700> can't assign resources (port)
Timecounters tick every 10.000 msec
IP Filter: v3.4.35 initialized. Default = pass all, Logging = enabled
Waiting 15 seconds for SCSI devices to settle
da0 at ahc0 bus 0 target 1 lun 0
da0: <SEAGATE ST318416N 0010> Fixed Direct Access SCSI-3 device
da0: 20.000MB/s transfers (20.000MHz, offset 15), Tagged Queueing Enabled
da0: 17522MB (35885168 512 byte sectors: 64H 32S/T 17522C)
SMP: AP CPU #1 Launched!
Mounting root from ufs:/dev/da0s1a
WARNING: / was not properly dismounted
ep0: promiscuous mode enabled
===============================================================================
oak# diff GENERIC OAK-TEST
25c25
< ident GENERIC
---
> ident OAK-TEST
29a30,35
> makeoptions DEBUG=-g #Build kernel with gdb(1) debug symbols
> options IPFILTER #ipfilter support
> options IPFILTER_LOG #ipfilter logging
> options SMP
> options DDB
> options KDB
38,40c44,46
< options NFSCLIENT # Network Filesystem Client
< options NFSSERVER # Network Filesystem Server
< options NFS_ROOT # NFS usable as /, requires NFSCLIENT
---
> #options NFSCLIENT # Network Filesystem Client
> #options NFSSERVER # Network Filesystem Server
> #options NFS_ROOT # NFS usable as /, requires NFSCLIENT
oak#
I also have the core file from the crash if any additional information can be obtained from that please let me know.
>How-To-Repeat:
Running these commands simultaneously will get a crash out within
45 minutes w/o debugging kernel on my systems. With the kdb/ddb stuff turned on it seems to be more on the order of hours rather than minutes. Also when the tcpdump starts and stops there is around 10 seconds or so where network traffic stops moving.
find / -type f -exec strings {} \; > /dev/null
dump 0sf 300000 - /home | gzip -9 > dumphome.gz
bzip2 <large image>.iso
scp <large image>.iso <another system>:/var/tmp
dump 0sf 300000 - / | gzip -9 > dumproot.gz
while 1
tcpdump -i fxp0 &
sleep 360
kill -9 `ps -auxww | grep tcpdump | grep -v grep | awk '{print $2}'`
sleep 60
end
>Fix:
>Release-Note:
>Audit-Trail:
>Unformatted:
More information about the freebsd-bugs
mailing list