kern/114370: 6.2 kernel with SMP options failed to dump core on
dual cpu board
Yong Rao
yrao at force10networks.com
Sat Jul 7 05:30:03 UTC 2007
>Number: 114370
>Category: kern
>Synopsis: 6.2 kernel with SMP options failed to dump core on dual cpu board
>Confidential: no
>Severity: critical
>Priority: high
>Responsible: freebsd-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: sw-bug
>Submitter-Id: current-users
>Arrival-Date: Sat Jul 07 05:30:02 GMT 2007
>Closed-Date:
>Last-Modified:
>Originator: Yong Rao
>Release: FreeBSD 6.2-RELEASE #2
>Organization:
Force10 Networks Inc
>Environment:
FreeBSD localhost.localdomain 6.2-RELEASE FreeBSD 6.2-RELEASE #2: Thu Jul 5 18:30:02 PDT 2007 root at localhost.localdomain:/usr/src/sys/i386/compile/PNICSMPPOLLDBG i386
The hardware settings.
We tried on 2 types of motherboards and CPUs. Both got the same problem.
a) the first settings
CPU: Dual Core AMD Opteron(tm) Processor 280 (2405.47-MHz 686-class CPU)
Origin = "AuthenticAMD" Id = 0x20f12 Stepping = 2
Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,P
GE ,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
Features2=0x1<SSE3>
AMD Features=0xe2500800<SYSCALL,NX,MMX+,FFXSR,LM,3DNow+,3DNow>
AMD Features2=0x3<LAHF,CMP>
Cores per package: 2
b) The 2nd setting.
CPU: Intel(R) Xeon(TM) CPU 2.80GHz (2800.11-MHz 686-class CPU)
Origin = "GenuineIntel" Id = 0xf29 Stepping = 9
Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,
PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,
HTT,TM,PBE>
Features2=0x4400<CNTX-ID,<b14>>
real memory = 2147418112 (2047 MB)
avail memory = 2096300032 (1999 MB)
ACPI APIC Table: <A M I OEMAPIC >
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs cpu0 (BSP):
APIC ID: 0
cpu1 (AP): APIC ID: 6
>Description:
Summary:
We recent found that the kernel compiled with SMP option has problem
in dumping the core when crash happens. This happened ONLY on the
motherboard that has either a duo-core cpu, or physically 2 cpus.
On ONE cpu board, this problem NEVER happens.
Phenomena:
When crash happens, just at the begining of core dump, it hang. The
following is the print.
mem dump: start address = 0x4352,len=0x30
Fatal trap 12: page fault while in kernel mode
cpuid = 1; apic id = 01
fault virtual address = 0x4352
fault code = supervisor read, page not present
instruction pointer = 0x20:0xc9e9fc92
stack pointer = 0x28:0xebdbdbdc
frame pointer = 0x28:0xebdbdbf8
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, def32 1, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 74231 (pnicdbg)
trap number = 12
panic: page fault
cpuid = 1
Uptime: 1d18h27m42s
Dumping 4030 MB (2 chunks)
chunk 0: 1MB (154 pages) ... ok
chunk 1: 4031MB (1031776 pages) (stopped and hung here)
>How-To-Repeat:
1) To isolate the issue. We used the kernel configuration file GENERIC,
but just added in the following 2 lines.
options SMP # Symmetric MultiProcessor Kernel
device apic # I/O APIC, required by SMP options
2) Rebuild the kernel by commands
cd /usr/src/sys/i386/conf
cp GENERIC TESTCONF
vi TESTCONF
config TESTCONF
cd ../compile/TESTCONF
make cleandepend; make clean
make depend; make
make install; sync
reboot
3) We loaded a simple device driver. Run a userland application, which sends
an ioctl request to kernel to read the illegal memary address. It caused the
crash of the kernel.
>Fix:
It is not fixed. But here are some debugging information.
I am attaching more debugs. After core dump hangs, I am able to get
into ddb. But the trace for the thread where core dump is running
does not look right.
I was able to isolate the dumper function – ad_dump
where it seems to be hanging.
Hopefully these debugs will give more clues. Please advise on what
more to look for. I am new to SMP and still coming up to speed.
The following debugs are taken after core dump hangs:
a) show pcpu
b) bt 100054 -- 100054 is the thread ID, which does the core dump
c) alltrace (in attached file “alltrace.txt”)
d) step and next command in debug
a) ---
db> show pcpu
cpuid = 1
curthread = 0xc9450d80: pid 16 "swi6: Giant taskq"
curpcb = 0xe7fe9d90
fpcurthread = none
idlethread = 0xc944f600: pid 10 "idle: cpu1"
APIC ID = 1
currentldt = 0x50
spin locks held:
db> show pcpu 0
cpuid = 0
curthread = 0xc96eb000: pid 894 "disablemeta"
curpcb = 0xe9b4fd90
fpcurthread = none
idlethread = 0xc944f780: pid 11 "idle: cpu0"
APIC ID = 0
currentldt = 0x50
spin locks held:
db>
b) ---
db> bt 100054
Tracing pid 894 tid 100054 td 0xc96eb000
sched_switch(3225858983,10,0,3382180240,3377804288,...) at sched_switch+331
_end(53,128,606517178,0,0,...) at 3920951848
_end(3377796632,3379230208,3379230476,3377796608,3379229696,...) at 3377796608
_end(3228796064,3228796064,3228796064,3228796064,3228796064,...) at 3379230208
null_method(0,0,0,0,0,...) at null_method
db>
c) ---
Please see attached file “alltrace.txt” for details.
d) trace ---
db> trace
Tracing pid 943 tid 100080 td 0xc998d000
kdb_enter(3228283314) at kdb_enter+43
panic(3390017776,3237235458,3956644984,21554,3389785424,...) at panic+295
mtpioctl(3389997312,3237235458,3389811456,3,3382235136,...) at mtpioctl+193
devfs_ioctl_f(3379694976,3237235458,3389811456,3376546688,3382235136) at devfs_5
ioctl(3382235136,3956645124) at ioctl+874
syscall(59,59,59,3216893976,134516360,...) at syscall+559
Xint0x80_syscall() at Xint0x80_syscall+31
--- syscall (54, FreeBSD ELF32, ioctl), eip = 672367343, esp = 3216893900, ebp -
db> s
[thread pid 943 tid 100080 ]
Stopped at kdb_enter+44: leave
db> s
[thread pid 943 tid 100080 ]
Stopped at kdb_enter+45: ret
db> s
[thread pid 943 tid 100080 ]
Stopped at panic+295: addl $4,%esp
db> s
[thread pid 943 tid 100080 ]
Stopped at panic+298: pushl $561
db> s
[thread pid 943 tid 100080 ]
Stopped at panic+303: pushl $3228283350
db> s
[thread pid 943 tid 100080 ]
Stopped at panic+308: pushl $0
db> s
[thread pid 943 tid 100080 ]
Stopped at panic+310: pushl $3228776128
db> s
[thread pid 943 tid 100080 ]
Stopped at panic+315: call _mtx_lock_spin_flags
db> s
[thread pid 943 tid 100080 ]
Stopped at _mtx_lock_spin_flags: pushl %ebp
db> s
[thread pid 943 tid 100080 ]
Stopped at _mtx_lock_spin_flags+1: movl %esp,%ebp
db> s
[thread pid 943 tid 100080 ]
Stopped at _mtx_lock_spin_flags+3: pushl %edi
db> s
[thread pid 943 tid 100080 ]
Stopped at _mtx_lock_spin_flags+4: pushl %esi
db> s
[thread pid 943 tid 100080 ]
Stopped at _mtx_lock_spin_flags+5: pushl %ebx
db> s
[thread pid 943 tid 100080 ]
Stopped at _mtx_lock_spin_flags+6: movl 8(%ebp),%esi
db> s
[thread pid 943 tid 100080 ]
Stopped at _mtx_lock_spin_flags+9: movl 20(%ebp),%edi
db> s
[thread pid 943 tid 100080 ]
Stopped at _mtx_lock_spin_flags+12: pushl %edi
db> s
[thread pid 943 tid 100080 ]
Stopped at _mtx_lock_spin_flags+13: pushl 16(%ebp)
db> s
[thread pid 943 tid 100080 ]
Stopped at _mtx_lock_spin_flags+16: movl 12(%ebp),%eax
db> s
[thread pid 943 tid 100080 ]
Stopped at _mtx_lock_spin_flags+19: orl $9,%eax
db> s
[thread pid 943 tid 100080 ]
Stopped at _mtx_lock_spin_flags+22: pushl %eax
db> s
[thread pid 943 tid 100080 ]
Stopped at _mtx_lock_spin_flags+23: pushl %esi
db> s
[thread pid 943 tid 100080 ]
Stopped at _mtx_lock_spin_flags+24: call witness_checkorder
db> n
After 22 instructions (0 loads, 0 stores),
[thread pid 943 tid 100080 ]
Stopped at witness_checkorder+1259: ret
db> n
KDB: stack backtrace:
hangs here ...
Patch attached with submission follows:
db> alltrace
Tracing command disablemeta pid 894 tid 100054 td 0xc96eb000
sched_switch(3225858983,10,0,3382180240,3377804288,...) at sched_switch+331
_end(53,128,606517178,0,0,...) at 3920951848
_end(3377796632,3379230208,3379230476,3377796608,3379229696,...) at 3377796608
_end(3228796064,3228796064,3228796064,3228796064,3228796064,...) at 3379230208
null_method(0,0,0,0,0,...) at null_method
Tracing command bash pid 884 tid 100058 td 0xc9991c00
sched_switch(3382254592,0,1) at sched_switch+331
mi_switch(1,0,0,3956742568,3226748917,...) at mi_switch+426
sleepq_switch(3377983504) at sleepq_switch+113
sleepq_wait_sig(3377983504) at sleepq_wait_sig+29
msleep(3377983504,0,345,3228302894,0) at msleep+510
ttysleep(3377983488,3377983504,345,3228302894,0,...) at ttysleep+33
ttread(3377983488,3956743356,0) at ttread+1167
ttyread(3377918464,3956743356,0,3228776192,0,...) at ttyread+47
giant_read(3377918464,3956743356,0) at giant_read+45
devfs_read_f(3379716600,3956743356,3387476352,0,3382254592) at devfs_read_f+98
dofileread(3382254592,0,3379716600,3956743356,4294967295,...) at dofileread+133
kern_readv(3382254592,0,3956743356,3217025183,1,...) at kern_readv+54
read(3382254592,3956743428) at read+69
syscall(59,59,59,0,3217025183,...) at syscall+559
Xint0x80_syscall() at Xint0x80_syscall+31
--- syscall (3, FreeBSD ELF32, read), eip = 135257039, esp = 3217025148, ebp = -
Tracing command bash pid 878 tid 100071 td 0xc96ea000
sched_switch(3379470336,0,1) at sched_switch+331
mi_switch(1,0,0,3920829452,3226748917,...) at mi_switch+426
sleepq_switch(3382776344) at sleepq_switch+113
sleepq_wait_sig(3382776344) at sleepq_wait_sig+29
msleep(3382776344,3382776448,348,3228297560,0) at msleep+510
kern_wait(3379470336,4294967295,3920829572,6,0) at kern_wait+1699
wait4(3379470336,3920829700) at wait4+42
syscall(59,59,59,894,0,...) at syscall+559
Xint0x80_syscall() at Xint0x80_syscall+31
--- syscall (7, FreeBSD ELF32, wait4), eip = 135042415, esp = 3217025340, ebp =-
Tracing command login pid 866 tid 100083 td 0xc9992780
sched_switch(3382257536,0,1) at sched_switch+331
mi_switch(1,0,0,3956829196,3226748917,...) at mi_switch+426
sleepq_switch(3385359504) at sleepq_switch+113
sleepq_wait_sig(3385359504) at sleepq_wait_sig+29
msleep(3385359504,3385359608,348,3228297560,0) at msleep+510
kern_wait(3382257536,884,3956829316,0,0) at kern_wait+1699
wait4(3382257536,3956829444) at wait4+42
syscall(59,59,59,582,1,...) at syscall+559
Xint0x80_syscall() at Xint0x80_syscall+31
--- syscall (7, FreeBSD ELF32, wait4), eip = 672093903, esp = 3217026396, ebp =-
Tracing command getty pid 865 tid 100090 td 0xc9c8c000
sched_switch(3385376768,0,1) at sched_switch+331
mi_switch(1,0,0,3957242260,3226748917,...) at mi_switch+426
sleepq_switch(3377985552) at sleepq_switch+113
sleepq_wait_sig(3377985552) at sleepq_wait_sig+29
msleep(3377985552,0,345,3228302894,0) at msleep+510
ttysleep(3377985536,3377985552,345,3228302894,0,...) at ttysleep+33
ttread(3377985536,3957243068,0) at ttread+1167
ttyread(3378041344,3957243068,0,3957242900,3226488553,...) at ttyread+47
scread(3378041344,3957243068,0,3228776192,0,...) at scread+34
giant_read(3378041344,3957243068,0) at giant_read+45
devfs_read_f(3379723360,3957243068,3376727936,0,3385376768) at devfs_read_f+98
dofileread(3385376768,0,3379723360,3957243068,4294967295,...) at dofileread+133
kern_readv(3385376768,0,3957243068,3217026591,1,...) at kern_readv+54
read(3385376768,3957243140) at read+69
syscall(59,59,59,0,134538624,...) at syscall+559
Xint0x80_syscall() at Xint0x80_syscall+31
--- syscall (3, FreeBSD ELF32, read), eip = 672428879, esp = 3217026556, ebp = -
Tracing command getty pid 864 tid 100068 td 0xc96ea480
sched_switch(3379471488,0,1) at sched_switch+331
mi_switch(1,0,0,3920865684,3226748917,...) at mi_switch+426
sleepq_switch(3377979408) at sleepq_switch+113
sleepq_wait_sig(3377979408) at sleepq_wait_sig+29
msleep(3377979408,0,345,3228302894,0) at msleep+510
ttysleep(3377979392,3377979408,345,3228302894,0,...) at ttysleep+33
ttread(3377979392,3920866492,0) at ttread+1167
ttyread(3378041088,3920866492,0,3920866324,3226488553,...) at ttyread+47
scread(3378041088,3920866492,0,3228776192,0,...) at scread+34
giant_read(3378041088,3920866492,0) at giant_read+45
devfs_read_f(3382342424,3920866492,3376727936,0,3379471488) at devfs_read_f+98
dofileread(3379471488,0,3382342424,3920866492,4294967295,...) at dofileread+133
kern_readv(3379471488,0,3920866492,3217026591,1,...) at kern_readv+54
read(3379471488,3920866564) at read+69
syscall(59,59,59,0,134538624,...) at syscall+559
Xint0x80_syscall() at Xint0x80_syscall+31
--- syscall (3, FreeBSD ELF32, read), eip = 672428879, esp = 3217026556, ebp = -
Tracing command getty pid 863 tid 100042 td 0xc9566480
sched_switch(3377882240,0,1) at sched_switch+331
mi_switch(1,0,0,3920550292,3226748917,...) at mi_switch+426
sleepq_switch(3378064400) at sleepq_switch+113
sleepq_wait_sig(3378064400) at sleepq_wait_sig+29
msleep(3378064400,0,345,3228302894,0) at msleep+510
ttysleep(3378064384,3378064400,345,3228302894,0,...) at ttysleep+33
ttread(3378064384,3920551100,0) at ttread+1167
ttyread(3378040832,3920551100,0,3920550932,3226488553,...) at ttyread+47
scread(3378040832,3920551100,0,3228776192,0,...) at scread+34
giant_read(3378040832,3920551100,0) at giant_read+45
devfs_read_f(3382344728,3920551100,3376727936,0,3377882240) at devfs_read_f+98
dofileread(3377882240,0,3382344728,3920551100,4294967295,...) at dofileread+133
kern_readv(3377882240,0,3920551100,3217026591,1,...) at kern_readv+54
read(3377882240,3920551172) at read+69
syscall(59,59,59,0,134538624,...) at syscall+559
Xint0x80_syscall() at Xint0x80_syscall+31
--- syscall (3, FreeBSD ELF32, read), eip = 672428879, esp = 3217026556, ebp = -
Tracing command getty pid 862 tid 100072 td 0xc9a12900
sched_switch(3382782208,0,1) at sched_switch+331
mi_switch(1,0,0,3957016980,3226748917,...) at mi_switch+426
sleepq_switch(3378063376) at sleepq_switch+113
sleepq_wait_sig(3378063376) at sleepq_wait_sig+29
msleep(3378063376,0,345,3228302894,0) at msleep+510
ttysleep(3378063360,3378063376,345,3228302894,0,...) at ttysleep+33
ttread(3378063360,3957017788,0) at ttread+1167
ttyread(3377924352,3957017788,0,3957017620,3226488553,...) at ttyread+47
scread(3377924352,3957017788,0,3228776192,0,...) at scread+34
giant_read(3377924352,3957017788,0) at giant_read+45
devfs_read_f(3382342640,3957017788,3376727936,0,3382782208) at devfs_read_f+98
dofileread(3382782208,0,3382342640,3957017788,4294967295,...) at dofileread+133
kern_readv(3382782208,0,3957017788,3217026591,1,...) at kern_readv+54
read(3382782208,3957017860) at read+69
syscall(59,59,59,0,134538624,...) at syscall+559
Xint0x80_syscall() at Xint0x80_syscall+31
--- syscall (3, FreeBSD ELF32, read), eip = 672428879, esp = 3217026556, ebp = -
Tracing command getty pid 861 tid 100079 td 0xc9992d80
sched_switch(3382259072,0,1) at sched_switch+331
mi_switch(1,0,0,3956877716,3226748917,...) at mi_switch+426
sleepq_switch(3378059280) at sleepq_switch+113
sleepq_wait_sig(3378059280) at sleepq_wait_sig+29
msleep(3378059280,0,345,3228302894,0) at msleep+510
ttysleep(3378059264,3378059280,345,3228302894,0,...) at ttysleep+33
ttread(3378059264,3956878524,0) at ttread+1167
ttyread(3377924608,3956878524,0,3956878356,3226488553,...) at ttyread+47
scread(3377924608,3956878524,0,3228776192,0,...) at scread+34
giant_read(3377924608,3956878524,0) at giant_read+45
devfs_read_f(3379719696,3956878524,3376727936,0,3382259072) at devfs_read_f+98
dofileread(3382259072,0,3379719696,3956878524,4294967295,...) at dofileread+133
kern_readv(3382259072,0,3956878524,3217026591,1,...) at kern_readv+54
read(3382259072,3956878596) at read+69
syscall(59,59,59,0,134538624,...) at syscall+559
Xint0x80_syscall() at Xint0x80_syscall+31
--- syscall (3, FreeBSD ELF32, read), eip = 672428879, esp = 3217026556, ebp = -
Tracing command getty pid 860 tid 100057 td 0xc9991d80
sched_switch(3382254976,0,1) at sched_switch+331
mi_switch(1,0,0,3956754836,3226748917,...) at mi_switch+426
sleepq_switch(3378058256) at sleepq_switch+113
sleepq_wait_sig(3378058256) at sleepq_wait_sig+29
msleep(3378058256,0,345,3228302894,0) at msleep+510
ttysleep(3378058240,3378058256,345,3228302894,0,...) at ttysleep+33
ttread(3378058240,3956755644,0) at ttread+1167
ttyread(3377924864,3956755644,0,3956755476,3226488553,...) at ttyread+47
scread(3377924864,3956755644,0,3228776192,0,...) at scread+34
giant_read(3377924864,3956755644,0) at giant_read+45
devfs_read_f(3379722928,3956755644,3376727936,0,3382254976) at devfs_read_f+98
dofileread(3382254976,0,3379722928,3956755644,4294967295,...) at dofileread+133
kern_readv(3382254976,0,3956755644,3217026591,1,...) at kern_readv+54
read(3382254976,3956755716) at read+69
syscall(59,59,59,0,134538624,...) at syscall+559
Xint0x80_syscall() at Xint0x80_syscall+31
--- syscall (3, FreeBSD ELF32, read), eip = 672428879, esp = 3217026556, ebp = -
Tracing command getty pid 859 tid 100088 td 0xc9c8c300
sched_switch(3385377536,0,1) at sched_switch+331
mi_switch(1,0,0,3957266836,3226748917,...) at mi_switch+426
sleepq_switch(3378062352) at sleepq_switch+113
sleepq_wait_sig(3378062352) at sleepq_wait_sig+29
msleep(3378062352,0,345,3228302894,0) at msleep+510
ttysleep(3378062336,3378062352,345,3228302894,0,...) at ttysleep+33
ttread(3378062336,3957267644,0) at ttread+1167
ttyread(3377925120,3957267644,0,3957267476,3226488553,...) at ttyread+47
scread(3377925120,3957267644,0,3228776192,0,...) at scread+34
giant_read(3377925120,3957267644,0) at giant_read+45
devfs_read_f(3382342928,3957267644,3376727936,0,3385377536) at devfs_read_f+98
dofileread(3385377536,0,3382342928,3957267644,4294967295,...) at dofileread+133
kern_readv(3385377536,0,3957267644,3217026591,1,...) at kern_readv+54
read(3385377536,3957267716) at read+69
syscall(59,59,59,0,134538624,...) at syscall+559
Xint0x80_syscall() at Xint0x80_syscall+31
--- syscall (3, FreeBSD ELF32, read), eip = 672428879, esp = 3217026556, ebp = -
Tracing command login pid 858 tid 100059 td 0xc9991a80
sched_switch(3382254208,0,1) at sched_switch+331
mi_switch(1,0,0,3956730892,3226748917,...) at mi_switch+426
sleepq_switch(3382249032) at sleepq_switch+113
sleepq_wait_sig(3382249032) at sleepq_wait_sig+29
msleep(3382249032,3382249136,348,3228297560,0) at msleep+510
kern_wait(3382254208,878,3956731012,0,0) at kern_wait+1699
wait4(3382254208,3956731140) at wait4+42
syscall(59,59,59,582,1,...) at syscall+559
Xint0x80_syscall() at Xint0x80_syscall+31
--- syscall (7, FreeBSD ELF32, wait4), eip = 672093903, esp = 3217026396, ebp =-
Tracing command cron pid 815 tid 100080 td 0xc9992c00
sched_switch(3382258688,0,1) at sched_switch+331
mi_switch(1,0,0,3956866108,3226749082,...) at mi_switch+426
sleepq_switch(3228779692) at sleepq_switch+113
sleepq_timedwait_sig(3228779692) at sleepq_timedwait_sig+30
msleep(3228779692,0,348,3228286296,60001,...) at msleep+468
kern_nanosleep(3382258688,3956866252,3956866244) at kern_nanosleep+171
nanosleep(3382258688,3956866308) at nanosleep+79
syscall(59,59,59,134545548,60,...) at syscall+559
Xint0x80_syscall() at Xint0x80_syscall+31
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 672337699, esp = 3217026268,-
Tracing command sendmail pid 808 tid 100065 td 0xc96ea900
sched_switch(3379472640,0,1) at sched_switch+331
mi_switch(1,0,0,3920903260,3226748917,...) at mi_switch+426
sleepq_switch(3382772836) at sleepq_switch+113
sleepq_wait_sig(3382772836) at sleepq_wait_sig+29
msleep(3382772836,3382772888,360,3228284168,0) at msleep+510
kern_sigsuspend(3379472640,0,0,0,0,...) at kern_sigsuspend+163
sigsuspend(3379472640,3920903428) at sigsuspend+51
syscall(59,59,59,0,3217018880,...) at syscall+559
Xint0x80_syscall() at Xint0x80_syscall+31
--- syscall (341, FreeBSD ELF32, sigsuspend), eip = 673683383, esp = 3217018860-
Tracing command sendmail pid 804 tid 100041 td 0xc9566600
sched_switch(3377882624,0,1) at sched_switch+331
mi_switch(1,0,0,3920562976,3226749082,...) at mi_switch+426
sleepq_switch(3229095940) at sleepq_switch+113
sleepq_timedwait_sig(3229095940,3229095940,5001,3920563008,582,...) at sleepq_t0
cv_timedwait_sig(3229095940,3229095904,5001) at cv_timedwait_sig+290
kern_select(3377882624,4,3217016064,0,0,...) at kern_select+1169
select(3377882624,3920563460) at select+68
syscall(59,59,59,1,1,...) at syscall+559
Xint0x80_syscall() at Xint0x80_syscall+31
--- syscall (93, FreeBSD ELF32, select), eip = 674198223, esp = 3217015820, ebp-
Tracing command sshd pid 798 tid 100082 td 0xc9992900
sched_switch(3382257920,0,1) at sched_switch+331
mi_switch(1,0,0,3956841260,3226748917,...) at mi_switch+426
sleepq_switch(3229095940) at sleepq_switch+113
sleepq_wait_sig(3229095940,582,3228776128,3228296630,771,...) at sleepq_wait_si9
cv_wait_sig(3229095940,3229095904) at cv_wait_sig+281
kern_select(3382257920,5,134723872,0,0,...) at kern_select+1191
select(3382257920,3956841732) at select+68
syscall(59,59,59,1,10,...) at syscall+559
Xint0x80_syscall() at Xint0x80_syscall+31
--- syscall (93, FreeBSD ELF32, select), eip = 674546383, esp = 3217023340, ebp-
Tracing command usbd pid 761 tid 100050 td 0xc96eb600
sched_switch(3379475968,0,1) at sched_switch+331
mi_switch(1,0,0,3921001248,3226749082,...) at mi_switch+426
sleepq_switch(3229095940) at sleepq_switch+113
sleepq_timedwait_sig(3229095940,3229095940,30001,3921001280,582,...) at sleepq_0
cv_timedwait_sig(3229095940,3229095904,30001) at cv_timedwait_sig+290
kern_select(3379475968,6,3217026464,3217026336,0,...) at kern_select+1169
select(3379475968,3921001732) at select+68
syscall(59,59,59,3217026336,30,...) at syscall+559
Xint0x80_syscall() at Xint0x80_syscall+31
--- syscall (93, FreeBSD ELF32, select), eip = 672379599, esp = 3217026252, ebp-
Tracing command smbd pid 735 tid 100078 td 0xc9a12000
sched_switch(3382779904,0,1) at sched_switch+331
mi_switch(1,0,0,3956943964,3226748917,...) at mi_switch+426
sleepq_switch(3385356876) at sleepq_switch+113
sleepq_wait_sig(3385356876) at sleepq_wait_sig+29
msleep(3385356876,3385356928,360,3228284168,0) at msleep+510
kern_sigsuspend(3382779904,1073746048,0,0,0,...) at kern_sigsuspend+163
sigsuspend(3382779904,3956944132) at sigsuspend+51
syscall(59,59,59,137408512,137408512,...) at syscall+559
Xint0x80_syscall() at Xint0x80_syscall+31
--- syscall (341, FreeBSD ELF32, sigsuspend), eip = 676951991, esp = 3217021452-
Tracing command smbd pid 714 tid 100086 td 0xc9992300
sched_switch(3382256384,0,1) at sched_switch+331
mi_switch(1,0,0,3956792108,3226748917,...) at mi_switch+426
sleepq_switch(3229095940) at sleepq_switch+113
sleepq_wait_sig(3229095940,582,3228776128,3228296630,771,...) at sleepq_wait_si9
cv_wait_sig(3229095940,3229095904) at cv_wait_sig+281
kern_select(3382256384,22,3217021808,0,0,...) at kern_select+1191
select(3382256384,3956792580) at select+68
syscall(59,59,59,137408512,0,...) at syscall+559
Xint0x80_syscall() at Xint0x80_syscall+31
--- syscall (93, FreeBSD ELF32, select), eip = 677466831, esp = 3217021372, ebp-
Tracing command nmbd pid 710 tid 100089 td 0xc9c8c180
sched_switch(3385377152,0,1) at sched_switch+331
mi_switch(1,0,0,3957254944,3226749082,...) at mi_switch+426
sleepq_switch(3229095940) at sleepq_switch+113
sleepq_timedwait_sig(3229095940,3229095940,10001,3957254976,582,...) at sleepq_0
cv_timedwait_sig(3229095940,3229095904,10001) at cv_timedwait_sig+290
kern_select(3385377152,13,3217025088,0,0,...) at kern_select+1169
select(3385377152,3957255428) at select+68
syscall(59,59,59,13,3217025088,...) at syscall+559
Xint0x80_syscall() at Xint0x80_syscall+31
--- syscall (93, FreeBSD ELF32, select), eip = 674271951, esp = 3217024796, ebp-
Tracing command syslogd pid 631 tid 100044 td 0xc9566180
sched_switch(3377881472,0,1) at sched_switch+331
mi_switch(1,0,0,3920526124,3226748917,...) at mi_switch+426
sleepq_switch(3229095940) at sleepq_switch+113
sleepq_wait_sig(3229095940,582,3228776128,3228296630,771,...) at sleepq_wait_si9
cv_wait_sig(3229095940,3229095904) at cv_wait_sig+281
kern_select(3377881472,9,134570080,0,0,...) at kern_select+1191
select(3377881472,3920526596) at select+68
syscall(59,59,59,134570084,134570080,...) at syscall+559
Xint0x80_syscall() at Xint0x80_syscall+31
--- syscall (93, FreeBSD ELF32, select), eip = 672445135, esp = 3217021484, ebp-
Tracing command devd pid 565 tid 100060 td 0xc9991900
sched_switch(3382253824,0,1) at sched_switch+331
mi_switch(1,0,0,3956718380,3226748917,...) at mi_switch+426
sleepq_switch(3229095940) at sleepq_switch+113
sleepq_wait_sig(3229095940,582,3228776128,3228296630,771,...) at sleepq_wait_si9
cv_wait_sig(3229095940,3229095904) at cv_wait_sig+281
kern_select(3382253824,5,3217025488,0,0,...) at kern_select+1191
select(3382253824,3956718852) at select+68
syscall(59,59,59,0,8,...) at syscall+559
Xint0x80_syscall() at Xint0x80_syscall+31
--- syscall (93, FreeBSD ELF32, select), eip = 134698999, esp = 3217025388, ebp-
Tracing command adjkerntz pid 143 tid 100064 td 0xc96eaa80
sched_switch(3379473024,0,1) at sched_switch+331
mi_switch(1,0,0,3920915548,3226748917,...) at mi_switch+426
sleepq_switch(3382772300) at sleepq_switch+113
sleepq_wait_sig(3382772300) at sleepq_wait_sig+29
msleep(3382772300,3382772352,360,3228284168,0) at msleep+510
kern_sigsuspend(3379473024,0,0,0,0,...) at kern_sigsuspend+163
sigsuspend(3379473024,3920915716) at sigsuspend+51
syscall(59,59,59,0,25200,...) at syscall+559
Xint0x80_syscall() at Xint0x80_syscall+31
--- syscall (341, FreeBSD ELF32, sigsuspend), eip = 671856567, esp = 3217026460-
Tracing command schedcpu pid 42 tid 100046 td 0xc951bd80
sched_switch(3377577344,0,1) at sched_switch+331
mi_switch(1,0,0,3919867052,3226749014,...) at mi_switch+426
sleepq_switch(3919867124,3228776128,0,3228292512,566,...) at sleepq_switch+113
sleepq_timedwait(3919867124) at sleepq_timedwait+34
msleep(3919867124,0,0,3228262727,1000) at msleep+489
schedcpu_thread(0,3919867192,3228776128,0,3228273492,...) at schedcpu_thread+38
fork_exit(3226695036,0,3919867192) at fork_exit+97
fork_trampoline() at fork_trampoline+8
--- trap 1, eip = 0, esp = 3919867244, ebp = 0 ---
Tracing command softdepflush pid 41 tid 100047 td 0xc951bc00
sched_switch(3377576960,0,1) at sched_switch+331
mi_switch(1,0,0,3919854756,3226749014,...) at mi_switch+426
sleepq_switch(3229151220,3228776128,0,3228292512,566,...) at sleepq_switch+113
sleepq_timedwait(3229151220) at sleepq_timedwait+34
msleep(3229151220,3229151168,68,3228349621,1000) at msleep+489
softdep_flush(0,3919854904,3228776128,0,3228273492,...) at softdep_flush+499
fork_exit(3227615092,0,3919854904) at fork_exit+97
fork_trampoline() at fork_trampoline+8
--- trap 1, eip = 0, esp = 3919854956, ebp = 0 ---
Tracing command vnlru pid 40 tid 100048 td 0xc951ba80
sched_switch(3377576576,0,1) at sched_switch+331
mi_switch(1,0,1,3919842460,3226749014,...) at mi_switch+426
sleepq_switch(3379465360,3228776128,0,3228292512,566,...) at sleepq_switch+113
sleepq_timedwait(3379465360) at sleepq_timedwait+34
msleep(3379465360,3229098336,592,3228312087,1000,3229098464) at msleep+489
vnlru_proc(0,3919842616,3228776128,0,3228273492,...) at vnlru_proc+223
fork_exit(3226951860,0,3919842616) at fork_exit+97
fork_trampoline() at fork_trampoline+8
--- trap 1, eip = 0, esp = 3919842668, ebp = 0 ---
Tracing command syncer pid 39 tid 100049 td 0xc951b900
sched_switch(3377576192,0,1) at sched_switch+331
mi_switch(1,0,1,3919830160,3226748861,...) at mi_switch+426
sleepq_switch(3228779036,3228776128,0,3228292512,529,...) at sleepq_switch+113
sleepq_wait(3228779036,0,3226599612,3229098400,8,...) at sleepq_wait+33
msleep(3228779036,0,104,3228312226,0) at msleep+525
sched_sync(0,3919830328,3228776128,0,3228273492,...) at sched_sync+777
fork_exit(3226957540,0,3919830328) at fork_exit+97
fork_trampoline() at fork_trampoline+8
--- trap 1, eip = 0, esp = 3919830380, ebp = 0 ---
Tracing command bufdaemon pid 38 tid 100027 td 0xc951b180
sched_switch(3377574272,0,1) at sched_switch+331
mi_switch(1,0,1,3919768756,3226749014,...) at mi_switch+426
sleepq_switch(3229097324,3228776128,0,3228292512,566,...) at sleepq_switch+113
sleepq_timedwait(3229097324) at sleepq_timedwait+34
msleep(3229097324,3229097344,68,3228307860,1000) at msleep+489
buf_daemon(0,3919768888,3228776128,0,3228273492,...) at buf_daemon+264
fork_exit(3226903868,0,3919768888) at fork_exit+97
fork_trampoline() at fork_trampoline+8
--- trap 1, eip = 0, esp = 3919768940, ebp = 0 ---
Tracing command pagezero pid 37 tid 100028 td 0xc951b000
sched_switch(3377573888,0,1) at sched_switch+331
mi_switch(1,0,0,3919756452,3226749014,...) at mi_switch+426
sleepq_switch(3229155172,3228776128,0,3228292512,566,...) at sleepq_switch+113
sleepq_timedwait(3229155172) at sleepq_timedwait+34
msleep(3229155172,3229153856,512,3228365548,300000,3229153856,0,3228365522,157)9
vm_pagezero(0,3919756600,3228776128,0,3228273492,...) at vm_pagezero+108
fork_exit(3227806548,0,3919756600) at fork_exit+97
fork_trampoline() at fork_trampoline+8
--- trap 1, eip = 0, esp = 3919756652, ebp = 0 ---
Tracing command idlepoll pid 36 tid 100029 td 0xc9454d80
sched_switch(3376762240,0,1) at sched_switch+331
mi_switch(1,0,0,3892346024,3226749014,...) at mi_switch+426
sleepq_switch(3228776544,3228776128,0,3228292512,566,...) at sleepq_switch+113
sleepq_timedwait(3228776544) at sleepq_timedwait+34
msleep(3228776544,0,0,3228281840,3000) at msleep+489
poll_idle(0,3892346168,3228776128,0,3228273492,...) at poll_idle+226
fork_exit(3226607484,0,3892346168) at fork_exit+97
fork_trampoline() at fork_trampoline+8
--- trap 1, eip = 0, esp = 3892346220, ebp = 0 ---
Tracing command vmdaemon pid 35 tid 100030 td 0xc9454c00
sched_switch(3376761856,0,1) at sched_switch+331
mi_switch(1,0,1,3892333716,3226748861,...) at mi_switch+426
sleepq_switch(3229153972,3228776128,0,3228292512,529,...) at sleepq_switch+113
sleepq_wait(3229153972,0,3226599551,3228776192,3228365165,...) at sleepq_wait+33
msleep(3229153972,0,104,3228307860,0) at msleep+525
vm_daemon(0,3892333880,3228776128,0,3228273492,...) at vm_daemon+54
fork_exit(3227802892,0,3892333880) at fork_exit+97
fork_trampoline() at fork_trampoline+8
--- trap 1, eip = 0, esp = 3892333932, ebp = 0 ---
Tracing command pagedaemon pid 34 tid 100031 td 0xc9454a80
sched_switch(3376761472,0,1) at sched_switch+331
mi_switch(1,0,0,3892321456,3226749014,...) at mi_switch+426
sleepq_switch(3229153904,3228776128,0,3228292512,566,...) at sleepq_switch+113
sleepq_timedwait(3229153904) at sleepq_timedwait+34
msleep(3229153904,3229153856,68,3228307860,5000) at msleep+489
vm_pageout(0,3892321592,3228776128,0,3228273492,...) at vm_pageout+608
fork_exit(3227802060,0,3892321592) at fork_exit+97
fork_trampoline() at fork_trampoline+8
--- trap 1, eip = 0, esp = 3892321644, ebp = 0 ---
Tracing command irq7: ppc0 pid 33 tid 100032 td 0xc9454900
fork_trampoline() at fork_trampoline
Tracing command swi0: sio pid 32 tid 100033 td 0xc9454780
sched_switch(3376760704,0,1) at sched_switch+331
mi_switch(1,0) at mi_switch+426
ithread_loop(3377944496,3892297016,3228776128,0,3228273492,...) at ithread_loop2
fork_exit(3226555672,3377944496,3892297016) at fork_exit+97
fork_trampoline() at fork_trampoline+8
--- trap 1, eip = 0, esp = 3892297068, ebp = 0 ---
Tracing command fdc0 pid 31 tid 100034 td 0xc9454600
sched_switch(3376760320,0,1) at sched_switch+331
mi_switch(1,0,0,3892284500,3226749014,...) at mi_switch+426
sleepq_switch(3377805884,3228776128,0,3228292512,566,...) at sleepq_switch+113
sleepq_timedwait(3377805884) at sleepq_timedwait+34
msleep(3377805884,3377806064,76,3228262727,1000) at msleep+489
fdc_worker(3377805824,3377806064,0,3228371674,1166) at fdc_worker+458
fdc_thread(3377805824,3892284728,3228776128,0,3228273492,...) at fdc_thread+84
fork_exit(3227861752,3377805824,3892284728) at fork_exit+97
fork_trampoline() at fork_trampoline+8
--- trap 1, eip = 0, esp = 3892284780, ebp = 0 ---
Tracing command irq1: atkbd0 pid 30 tid 100035 td 0xc9454480
sched_switch(3376759936,0,1) at sched_switch+331
mi_switch(1,0) at mi_switch+426
ithread_loop(3377944864,3892272440,3228776128,0,3228273492,...) at ithread_loop2
fork_exit(3226555672,3377944864,3892272440) at fork_exit+97
fork_trampoline() at fork_trampoline+8
--- trap 1, eip = 0, esp = 3892272492, ebp = 0 ---
Tracing command irq24: bge1 pid 29 tid 100036 td 0xc9566d80
sched_switch(3377884544,0,1) at sched_switch+331
mi_switch(1,0) at mi_switch+426
ithread_loop(3377837568,3920624952,3228776128,0,3228273492,...) at ithread_loop2
fork_exit(3226555672,3377837568,3920624952) at fork_exit+97
fork_trampoline() at fork_trampoline+8
--- trap 1, eip = 0, esp = 3920625004, ebp = 0 ---
Tracing command irq27: bge0 pid 28 tid 100037 td 0xc9566c00
sched_switch(3377884160,0,1) at sched_switch+331
mi_switch(1,0) at mi_switch+426
ithread_loop(3377838352,3920612664,3228776128,0,3228273492,...) at ithread_loop2
fork_exit(3226555672,3377838352,3920612664) at fork_exit+97
fork_trampoline() at fork_trampoline+8
--- trap 1, eip = 0, esp = 3920612716, ebp = 0 ---
Tracing command irq15: ata1 pid 27 tid 100017 td 0xc9450900
fork_trampoline() at fork_trampoline
Tracing command irq14: ata0 pid 26 tid 100018 td 0xc9450780
fork_trampoline() at fork_trampoline
Tracing command irq16: atapci0 pid 25 tid 100019 td 0xc9450600
sched_switch(3376743936,0,1) at sched_switch+331
mi_switch(1,0) at mi_switch+426
ithread_loop(3377734416,3892161848,3228776128,0,3228273492,...) at ithread_loop2
fork_exit(3226555672,3377734416,3892161848) at fork_exit+97
fork_trampoline() at fork_trampoline+8
--- trap 1, eip = 0, esp = 3892161900, ebp = 0 ---
Tracing command usb1 pid 24 tid 100020 td 0xc9450480
sched_switch(3376743552,0,1) at sched_switch+331
mi_switch(1,0,1,3892149420,3226749014,...) at mi_switch+426
sleepq_switch(3377725968,3228776128,0,3228292512,566,...) at sleepq_switch+113
sleepq_timedwait(3377725968) at sleepq_timedwait+34
msleep(3377725968,0,92,3228259087,60000,3377742272) at msleep+489
usb_event_thread(3377742272,3892149560,3228776128,0,3228273492,...) at usb_even2
fork_exit(3226327968,3377742272,3892149560) at fork_exit+97
fork_trampoline() at fork_trampoline+8
--- trap 1, eip = 0, esp = 3892149612, ebp = 0 ---
Tracing command usbtask pid 23 tid 100021 td 0xc9450300
sched_switch(3376743168,0,1) at sched_switch+331
mi_switch(1,0,1,3892137140,3226748861,...) at mi_switch+426
sleepq_switch(3228766148,3228776128,0,3228292512,529,...) at sleepq_switch+113
sleepq_wait(3228766148,0,3226599551,3228776192,3228259073,...) at sleepq_wait+33
msleep(3228766148,0,92,3228259094,0,...) at msleep+525
usb_task_thread(0,3892137272,3228776128,0,3228273492,...) at usb_task_thread+71
fork_exit(3226328112,0,3892137272) at fork_exit+97
fork_trampoline() at fork_trampoline+8
--- trap 1, eip = 0, esp = 3892137324, ebp = 0 ---
Tracing command usb0 pid 22 tid 100022 td 0xc9450180
sched_switch(3376742784,0,1) at sched_switch+331
mi_switch(1,0,1,3892124844,3226749014,...) at mi_switch+426
sleepq_switch(3377709584,3228776128,0,3228292512,566,...) at sleepq_switch+113
sleepq_timedwait(3377709584) at sleepq_timedwait+34
msleep(3377709584,0,92,3228259087,60000,3377645696) at msleep+489
usb_event_thread(3377645696,3892124984,3228776128,0,3228273492,...) at usb_even2
fork_exit(3226327968,3377645696,3892124984) at fork_exit+97
fork_trampoline() at fork_trampoline+8
--- trap 1, eip = 0, esp = 3892125036, ebp = 0 ---
Tracing command irq19: ohci0 ohci1 pid 21 tid 100023 td 0xc951b780
fork_trampoline() at fork_trampoline
Tracing command irq9: acpi0 pid 20 tid 100024 td 0xc951b600
fork_trampoline() at fork_trampoline
Tracing command swi6: task queue pid 19 tid 100025 td 0xc951b480
sched_switch(3377575040,0,1) at sched_switch+331
mi_switch(1,0) at mi_switch+426
ithread_loop(3376511392,3919793464,3228776128,0,3228273492,...) at ithread_loop2
fork_exit(3226555672,3376511392,3919793464) at fork_exit+97
fork_trampoline() at fork_trampoline+8
--- trap 1, eip = 0, esp = 3919793516, ebp = 0 ---
Tracing command kqueue taskq pid 9 tid 100026 td 0xc951b300
sched_switch(3377574656,0,1) at sched_switch+331
mi_switch(1,0,0,3919781032,3226748861,...) at mi_switch+426
sleepq_switch(3377065088,3228776128,0,3228292512,529,...) at sleepq_switch+113
sleepq_wait(3377065088,0,3228293171,73,8,...) at sleepq_wait+33
msleep(3377065088,3377065116,0,3228262727,0) at msleep+525
taskqueue_thread_loop(3228769632,3919781176,3228776128,0,3228273492,...) at tas0
fork_exit(3226753588,3228769632,3919781176) at fork_exit+97
fork_trampoline() at fork_trampoline+8
--- trap 1, eip = 0, esp = 3919781228, ebp = 0 ---
Tracing command acpi_task_2 pid 8 tid 100008 td 0xc944f300
sched_switch(3376739072,0,1) at sched_switch+331
mi_switch(1,0,0,3892014248,3226748861,...) at mi_switch+426
sleepq_switch(3377065344,3228776128,0,3228292512,529,...) at sleepq_switch+113
sleepq_wait(3377065344,0,3228293171,73,8,...) at sleepq_wait+33
msleep(3377065344,3377065372,0,3228262727,0) at msleep+525
taskqueue_thread_loop(3230234752,3892014392,3228776128,0,3228273492,...) at tas0
fork_exit(3226753588,3230234752,3892014392) at fork_exit+97
fork_trampoline() at fork_trampoline+8
--- trap 1, eip = 0, esp = 3892014444, ebp = 0 ---
Tracing command acpi_task_1 pid 7 tid 100009 td 0xc944f180
sched_switch(3376738688,0,1) at sched_switch+331
mi_switch(1,0,0,3892001960,3226748861,...) at mi_switch+426
sleepq_switch(3377065344,3228776128,0,3228292512,529,...) at sleepq_switch+113
sleepq_wait(3377065344,0,3228293171,73,8,...) at sleepq_wait+33
msleep(3377065344,3377065372,0,3228262727,0) at msleep+525
taskqueue_thread_loop(3230234752,3892002104,3228776128,0,3228273492,...) at tas0
fork_exit(3226753588,3230234752,3892002104) at fork_exit+97
fork_trampoline() at fork_trampoline+8
--- trap 1, eip = 0, esp = 3892002156, ebp = 0 ---
Tracing command acpi_task_0 pid 6 tid 100010 td 0xc944f000
sched_switch(3376738304,0,1) at sched_switch+331
mi_switch(1,0,0,3891989672,3226748861,...) at mi_switch+426
sleepq_switch(3377065344,3228776128,0,3228292512,529,...) at sleepq_switch+113
sleepq_wait(3377065344,0,3228293171,73,3377600608,...) at sleepq_wait+33
msleep(3377065344,3377065372,0,3228262727,0) at msleep+525
taskqueue_thread_loop(3230234752,3891989816,3228776128,0,3228273492,...) at tas0
fork_exit(3226753588,3230234752,3891989816) at fork_exit+97
fork_trampoline() at fork_trampoline+8
--- trap 1, eip = 0, esp = 3891989868, ebp = 0 ---
Tracing command swi2: cambio pid 18 tid 100011 td 0xc9454300
sched_switch(3376759552,0,1) at sched_switch+331
mi_switch(1,0) at mi_switch+426
ithread_loop(3376511456,3892260152,3228776128,0,3228273492,...) at ithread_loop2
fork_exit(3226555672,3376511456,3892260152) at fork_exit+97
fork_trampoline() at fork_trampoline+8
--- trap 1, eip = 0, esp = 3892260204, ebp = 0 ---
Tracing command swi5: + pid 17 tid 100012 td 0xc9454180
fork_trampoline() at fork_trampoline
Tracing command thread taskq pid 5 tid 100013 td 0xc9454000
sched_switch(3376758784,0,1) at sched_switch+331
mi_switch(1,0,0,3892235432,3226748861,...) at mi_switch+426
sleepq_switch(3377103104,3228776128,0,3228292512,529,...) at sleepq_switch+113
sleepq_wait(3377103104,0,3228293171,73,8,...) at sleepq_wait+33
msleep(3377103104,3377103132,0,3228262727,0) at msleep+525
taskqueue_thread_loop(3228801864,3892235576,3228776128,0,3228273492,...) at tas0
fork_exit(3226753588,3228801864,3892235576) at fork_exit+97
fork_trampoline() at fork_trampoline+8
--- trap 1, eip = 0, esp = 3892235628, ebp = 0 ---
Tracing command swi6: Giant taskq pid 16 tid 100014 td 0xc9450d80
kdb_enter(3228375620) at kdb_enter+43
scgetc(3229266944,2) at scgetc+1348
sckbdevent(3377072384,0,3229266944) at sckbdevent+456
kbdmux_intr(3377072384,0,3377105560,3892223160,3226752756,...) at kbdmux_intr+28
kbdmux_kbd_intr(3377072384,1) at kbdmux_kbd_intr+26
taskqueue_run(3377103360,3892223196,3226555506,0,3376511600,...) at taskqueue_r0
taskqueue_swi_giant_run(0) at taskqueue_swi_giant_run+14
ithread_execute_handlers(3377065984,3377103232) at ithread_execute_handlers+206
ithread_loop(3376511600,3892223288,3228776128,0,3228273492,...) at ithread_loop9
fork_exit(3226555672,3376511600,3892223288) at fork_exit+97
fork_trampoline() at fork_trampoline+8
--- trap 1, eip = 0, esp = 3892223340, ebp = 0 ---
Tracing command yarrow pid 15 tid 100015 td 0xc9450c00
sched_switch(3376745472,0,1) at sched_switch+331
mi_switch(1,0,0,3892210848,3226749014,...) at mi_switch+426
sleepq_switch(3228757952,3228776128,0,3228292512,566,...) at sleepq_switch+113
sleepq_timedwait(3228757952) at sleepq_timedwait+34
msleep(3228757952,0,0,3228262727,100) at msleep+489
random_kthread(0,3892211000,3228776128,0,3228273492,...) at random_kthread+369
fork_exit(3226233400,0,3892211000) at fork_exit+97
fork_trampoline() at fork_trampoline+8
--- trap 1, eip = 0, esp = 3892211052, ebp = 0 ---
Tracing command g_down pid 4 tid 100016 td 0xc9450a80
sched_switch(3376745088,0,1) at sched_switch+331
mi_switch(1,0,0,3892198556,3226749014,...) at mi_switch+426
sleepq_switch(3228767432,3228776128,0,3228292512,566,...) at sleepq_switch+113
sleepq_timedwait(3228767432) at sleepq_timedwait+34
msleep(3228767432,3228767144,588,3228262727,100) at msleep+489
g_io_schedule_down(3376745088) at g_io_schedule_down+86
g_down_procbody(0,3892198712,3228776128,0,3228273492,...) at g_down_procbody+70
fork_exit(3226426420,0,3892198712) at fork_exit+97
fork_trampoline() at fork_trampoline+8
--- trap 1, eip = 0, esp = 3892198764, ebp = 0 ---
Tracing command g_up pid 3 tid 100000 td 0xc9450000
sched_switch(3376742400,0,1) at sched_switch+331
mi_switch(1,0,0,3892112544,3226749014,...) at mi_switch+426
sleepq_switch(3228767428,3228776128,0,3228292512,566,...) at sleepq_switch+113
sleepq_timedwait(3228767428) at sleepq_timedwait+34
msleep(3228767428,3228767208,588,3228262727,100) at msleep+489
g_io_schedule_up(3376742400) at g_io_schedule_up+204
g_up_procbody(0,3892112696,3228776128,0,3228273492,...) at g_up_procbody+70
fork_exit(3226426344,0,3892112696) at fork_exit+97
fork_trampoline() at fork_trampoline+8
--- trap 1, eip = 0, esp = 3892112748, ebp = 0 ---
Tracing command g_event pid 2 tid 100001 td 0xc944fd80
sched_switch(3376741760,0,1) at sched_switch+331
mi_switch(1,0,0,3892100260,3226749014,...) at mi_switch+426
sleepq_switch(3228767420,3228776128,0,3228292512,566,...) at sleepq_switch+113
sleepq_timedwait(3228767420) at sleepq_timedwait+34
msleep(3228767420,0,76,3228262727,100) at msleep+489
g_event_procbody(0,3892100408,3228776128,0,3228273492,...) at g_event_procbody+0
fork_exit(3226426496,0,3892100408) at fork_exit+97
fork_trampoline() at fork_trampoline+8
--- trap 1, eip = 0, esp = 3892100460, ebp = 0 ---
Tracing command swi3: vm pid 14 tid 100002 td 0xc944fc00
fork_trampoline() at fork_trampoline
Tracing command swi4: clock sio pid 13 tid 100003 td 0xc944fa80
sched_switch(3376740992,0,1) at sched_switch+331
mi_switch(1,0) at mi_switch+426
ithread_loop(3376511136,3892075832,3228776128,0,3228273492,...) at ithread_loop2
fork_exit(3226555672,3376511136,3892075832) at fork_exit+97
fork_trampoline() at fork_trampoline+8
--- trap 1, eip = 0, esp = 3892075884, ebp = 0 ---
Tracing command swi1: net pid 12 tid 100004 td 0xc944f900
sched_switch(3376740608,0,1) at sched_switch+331
mi_switch(1,0) at mi_switch+426
ithread_loop(3376511152,3892063544,3228776128,0,3228273492,...) at ithread_loop2
fork_exit(3226555672,3376511152,3892063544) at fork_exit+97
fork_trampoline() at fork_trampoline+8
--- trap 1, eip = 0, esp = 3892063596, ebp = 0 ---
Tracing command idle: cpu0 pid 11 tid 100005 td 0xc944f780
sched_switch(3376740224,3376743936,6) at sched_switch+331
mi_switch(6,3376743936,3376744280,3376743936,3892050952,...) at mi_switch+426
maybe_preempt(3376743936) at maybe_preempt+196
sched_add(3376743936,4,3376743936,3377734416,3377063168,...) at sched_add+240
setrunqueue(3376743936,4) at setrunqueue+99
intr_event_schedule_thread(3377063168) at intr_event_schedule_thread+163
intr_execute_handlers(3376699000,3892051072,16,3892051140,3227972099,...) at in0
lapic_handle_intr(50) at lapic_handle_intr+46
Xapic_isr1() at Xapic_isr1+51
--- interrupt, eip = 3230175589, esp = 3892051140, ebp = 3892051140 ---
acpi_cpu_c1(3228776128,0,3892051172,1,4294967294,...) at acpi_cpu_c1+5
acpi_cpu_idle(3892051196,3226552629,0,3376736888,3892051236,...) at acpi_cpu_id8
cpu_idle(0,3376736888,3892051236,3226552149,0,...) at cpu_idle+40
idle_proc(0,3892051256,3228776128,0,3228273492,...) at idle_proc+77
fork_exit(3226552552,0,3892051256) at fork_exit+97
fork_trampoline() at fork_trampoline+8
--- trap 1, eip = 0, esp = 3892051308, ebp = 0 ---
Tracing command idle: cpu1 pid 10 tid 100006 td 0xc944f600
sched_switch(3376739840,0,1) at sched_switch+331
mi_switch(1,0,3228776128,0,3228273740,...) at mi_switch+426
idle_proc(0,3892038968,3228776128,0,3228273492,...) at idle_proc+123
fork_exit(3226552552,0,3892038968) at fork_exit+97
fork_trampoline() at fork_trampoline+8
--- trap 1, eip = 0, esp = 3892039020, ebp = 0 ---
Tracing command init pid 1 tid 100007 td 0xc944f480
sched_switch(3376739456,0,1) at sched_switch+331
mi_switch(1,0,0,3892026380,3226748917,...) at mi_switch+426
sleepq_switch(3376754688) at sleepq_switch+113
sleepq_wait_sig(3376754688) at sleepq_wait_sig+29
msleep(3376754688,3376754792,348,3228297560,0) at msleep+510
kern_wait(3376739456,4294967295,3892026500,0,0) at kern_wait+1699
wait4(3376739456,3892026628) at wait4+42
syscall(59,59,59,0,0,...) at syscall+559
Xint0x80_syscall() at Xint0x80_syscall+31
--- syscall (7, FreeBSD ELF32, wait4), eip = 134557511, esp = 3217026428, ebp =-
Tracing command swapper pid 0 tid 0 td 0xc07317e0
sched_switch(3228768224,0,1) at sched_switch+331
mi_switch(1,0) at mi_switch+426
scheduler(0,12708864,12705792,0,3225640485,...) at scheduler+385
mi_startup() at mi_startup+150
begin() at begin+44
db>
>Release-Note:
>Audit-Trail:
>Unformatted:
More information about the freebsd-bugs
mailing list