kern/118044: panic: spin lock held too long
Per olof Ljungmark
peo at intersonic.se
Wed Nov 14 07:20:03 PST 2007
>Number: 118044
>Category: kern
>Synopsis: panic: spin lock held too long
>Confidential: no
>Severity: critical
>Priority: high
>Responsible: freebsd-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: sw-bug
>Submitter-Id: current-users
>Arrival-Date: Wed Nov 14 15:20:01 UTC 2007
>Closed-Date:
>Last-Modified:
>Originator: Per olof Ljungmark
>Release: RELENG_7
>Organization:
Intersonic AB
>Environment:
%uname -a
FreeBSD ruben.i.inter-sonic.com 7.0-BETA2 FreeBSD 7.0-BETA2 #0: Tue Nov 13 14:58:10 CET 2007 peo at ruben.i.inter-sonic.com:/usr/obj/usr/src/sys/SMP-DBG i386
>Description:
Hi,
We have several all HP boxes running 7-STABLE (but the behaviour also
showed in 7-CURRENT, it is not new) that can be reliadbly crashed by
accessing them remotely from a remote network over a IPSec link running
on our firewall. Today I've built a debug kernel and caught the attached
information. Please let me know what more I can supply.
Among the affected units are ProLiant DL380G2, G3, G4, Workstations
XW600, XW6200 running SMP kernels. Not affected are DL360G3 from what
I've seen so far. Problem only shows with SMP kernels.
Kernels are GENERIC + SCHED_ULE
possibly connected to
http://www.freebsd.org/cgi/query-pr.cgi?pr=117913
================================
Copyright (c) 1992-2007 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 7.0-BETA2 #0: Tue Nov 13 14:58:10 CET 2007
peo at ruben.i.inter-sonic.com:/usr/obj/usr/src/sys/SMP-DBG
WARNING: WITNESS option enabled, expect reduced performance.
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Xeon(TM) CPU 3.20GHz (3189.72-MHz 686-class CPU)
Origin = "GenuineIntel" Id = 0xf25 Stepping = 5
Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0x4400<CNXT-ID,xTPR>
Logical CPUs per core: 2
real memory = 2147418112 (2047 MB)
avail memory = 2091864064 (1994 MB)
ACPI APIC Table: <COMPAQ PLACER >
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
cpu0 (BSP): APIC ID: 0
cpu1 (AP): APIC ID: 1
cpu2 (AP): APIC ID: 6
cpu3 (AP): APIC ID: 7
ioapic0: Changing APIC ID to 1
ioapic0 <Version 2.0> irqs 0-23 on motherboard
kbd1 at kbdmux0
ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413)
acpi0: <COMPAQ CPQ0054> on motherboard
acpi0: [ITHREAD]
acpi0: Power Button (fixed)
acpi0: reservation of 0, a0000 (3) failed
acpi0: reservation of 100000, 7ff00000 (3) failed
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0xf808-0xf80b on acpi0
cpu0: <ACPI CPU> on acpi0
p4tcc0: <CPU Frequency Thermal Control> on cpu0
cpu1: <ACPI CPU> on acpi0
p4tcc1: <CPU Frequency Thermal Control> on cpu1
cpu2: <ACPI CPU> on acpi0
p4tcc2: <CPU Frequency Thermal Control> on cpu2
cpu3: <ACPI CPU> on acpi0
p4tcc3: <CPU Frequency Thermal Control> on cpu3
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
agp0: <Intel E7505 host to AGP bridge> on hostb0
pcib1: <PCI-PCI bridge> mem 0xf8000000-0xfbffffff at device 1.0 on pci0
pci1: <PCI bus> on pcib1
vgapci0: <VGA-compatible display> mem
0xf1000000-0xf1ffffff,0xe8000000-0xefffffff irq 16 at device 0.0 on pci1
uhci0: <Intel 82801DB (ICH4) USB controller USB-A> port 0x2440-0x245f
irq 16 at device 29.0 on pci0
uhci0: [GIANT-LOCKED]
uhci0: [ITHREAD]
usb0: <Intel 82801DB (ICH4) USB controller USB-A> on uhci0
usb0: USB revision 1.0
uhub0: <Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb0
uhub0: 2 ports with 2 removable, self powered
uhci1: <Intel 82801DB (ICH4) USB controller USB-B> port 0x2460-0x247f
irq 19 at device 29.1 on pci0
uhci1: [GIANT-LOCKED]
uhci1: [ITHREAD]
usb1: <Intel 82801DB (ICH4) USB controller USB-B> on uhci1
usb1: USB revision 1.0
uhub1: <Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb1
uhub1: 2 ports with 2 removable, self powered
ehci0: <Intel 82801DB/L/M (ICH4) USB 2.0 controller> mem
0xf0500000-0xf05003ff irq 23 at device 29.7 on pci0
ehci0: [GIANT-LOCKED]
ehci0: [ITHREAD]
usb2: EHCI version 1.0
usb2: wrong number of companions (3 != 2)
usb2: companion controllers, 2 ports each: usb0 usb1
usb2: <Intel 82801DB/L/M (ICH4) USB 2.0 controller> on ehci0
usb2: USB revision 2.0
uhub2: <Intel EHCI root hub, class 9/0, rev 2.00/1.00, addr 1> on usb2
uhub2: 6 ports with 6 removable, self powered
pcib2: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci5: <ACPI PCI bus> on pcib2
pci0:5:2:0: bad VPD cksum, remain 72
bge0: <Broadcom Gigabit Ethernet Controller, ASIC rev. 0x1002> mem
0xf0400000-0xf040ffff irq 21 at device 2.0 on pci5
miibus0: <MII bus> on bge0
brgphy0: <BCM5703 10/100/1000baseTX PHY> PHY 1 on miibus0
brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT,
1000baseT-FDX, auto
bge0: Ethernet address: 00:0d:9d:57:71:9a
bge0: [ITHREAD]
aac0: <Adaptec SCSI RAID 2120S> mem 0xe4000000-0xe7ffffff irq 18 at
device 10.0 on pci5
aac0: New comm. interface enabled
aac0: [ITHREAD]
aac0: Adaptec Raid Controller 2.0.0-1
aacp0: <SCSI Passthrough Bus> on aac0
ahd0: <Adaptec AIC7902 Ultra320 SCSI adapter> port
0x1000-0x10ff,0x1400-0x14ff mem 0xf0410000-0xf0411fff irq 19 at device
12.0 on pci5
ahd0: [ITHREAD]
aic7902: Ultra320 Wide Channel A, SCSI Id=7, PCI 33 or 66Mhz, 512 SCBs
ahd1: <Adaptec AIC7902 Ultra320 SCSI adapter> port
0x1800-0x18ff,0x1c00-0x1cff mem 0xf0412000-0xf0413fff irq 22 at device
12.1 on pci5
ahd1: [ITHREAD]
aic7902: Ultra320 Wide Channel B, SCSI Id=7, PCI 33 or 66Mhz, 512 SCBs
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel ICH4 UDMA100 controller> port
0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x24a0-0x24af irq 18 at device 31.1
on pci0
ata0: <ATA channel 0> on atapci0
ata0: [ITHREAD]
ata1: <ATA channel 1> on atapci0
ata1: [ITHREAD]
pci0: <multimedia, audio> at device 31.5 (no driver attached)
acpi_button0: <Power Button> on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
atkbd0: [ITHREAD]
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: [ITHREAD]
psm0: model Generic PS/2 mouse, device ID 0
sio0: <Standard PC COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
sio0: type 16550A, console
sio0: [FILTER]
sio1: <Standard PC COM port> port 0x2f8-0x2ff irq 3 on acpi0
sio1: type 16550A
sio1: [FILTER]
fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0
fdc0: [FILTER]
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
pmtimer0 on isa0
orm0: <ISA Option ROM> at iomem 0xea800-0xeffff pnpid ORM0000 on isa0
ppc0: <Parallel port> at port 0x378-0x37f irq 7 on isa0
ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode
ppc0: FIFO with 16/16/13 bytes threshold
ppbus0: <Parallel port bus> on ppc0
ppi0: <Parallel I/O> on ppbus0
plip0: <PLIP network interface> on ppbus0
lpt0: <Printer> on ppbus0
lpt0: Interrupt-driven port
ppc0: [GIANT-LOCKED]
ppc0: [ITHREAD]
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
Timecounters tick every 1.000 msec
acd0: CDROM <HL-DT-ST CD-ROM GCR-8482B/1.03> at ata1-master PIO4
Waiting 5 seconds for SCSI devices to settle
aacd0: <RAID 1 (Mirror)> on aac0
aacd0: 34970MB (71619584 sectors)
SMP: AP CPU #3 Launched!
SMP: AP CPU #1 Launched!
SMP: AP CPU #2 Launched!
WARNING: WITNESS option enabled, expect reduced performance.
Trying to mount root from ufs:/dev/aacd0s1a
WARNING: / was not properly dismounted
==========================
rtfree: 0xc5874618 has 1 refs
ssppiinn lloocckk 00xxcc5507fc28aa0500 ((ttuurrnnssttiillee lloocckk))
hheelldd bbyy 00xxcc55111197868600 ((ttiidd 110000002154)) ttoooo lloonngg
panic: spin lock held too long
cpuid = 1
KDB: enter: panic
[thread pid 16 tid 100014 ]
Stopped at kdb_enter+0x32: leave
db>
db> ps
pid ppid pgrp uid state wmesg wchan cmd
22686 22684 7399 0 R+ sed
22684 22651 7399 0 S+ wait 0xcb49eaa0 sh
22683 22205 22683 0 S+ ttywri 0xc5542c38 cat
22651 20101 7399 0 S+ piperd 0xc5bfb318 sh
22205 22198 22205 0 S+ pause 0xc6ef05b0 csh
22198 21992 22198 1001 S+ wait 0xc6eeb550 su
21992 21991 21992 1001 Ss+ pause 0xc5c9bb00 csh
21991 21982 21982 1001 L *rtentry 0xc57c8a50 sshd
21982 705 21982 0 Ss sbwait 0xc7600248 sshd
20101 20097 7399 0 S+ wait 0xcb4a0000 gmake
20097 20096 7399 0 S+ wait 0xc6eeb7f8 sh
20096 84323 7399 0 S+ wait 0xc559daa0 gmake
84323 84322 7399 0 S+ wait 0xcb4a0d48 sh
84322 84318 7399 0 S+ wait 0xca6ef2a8 gmake
84318 84317 7399 0 S+ wait 0xc5c20d48 sh
84317 84316 7399 0 S+ wait 0xc6eefaa0 gmake
84316 84315 7399 0 S+ wait 0xc5bc1d48 gmake
84315 20720 7399 0 S+ wait 0xc7224550 sh
20720 20719 7399 0 S+ wait 0xc5633000 make
--More--
20719 20703 7399 0 S+ wait 0xc5c97550 sh
20703 25073 7399 0 S+ wait 0xc5c87aa0 make
86175 85857 86175 0 S+ ttyin 0xc5528410 csh
85857 84932 85857 1001 S+ wait 0xc57ca7f8 su
84932 84911 84932 1001 Ss+ pause 0xc55d1308 csh
84911 83121 83121 1001 S select 0xc0c03cbc sshd
83121 705 83121 0 Ss sbwait 0xc57126ec sshd
25073 7399 7399 0 S+ wait 0xc5c97d48 sh
7399 810 7399 0 S+ wait 0xc562aaa0 make
810 809 810 0 S+ pause 0xc5633308 csh
809 805 809 1001 S+ wait 0xc5a38550 su
805 804 805 1001 Ss+ pause 0xc5a3c060 csh
804 801 801 1001 S select 0xc0c03cbc sshd
801 705 801 0 Ss sbwait 0xc5712a04 sshd
793 1 793 0 Ss+ ttyin 0xc535d810 login
792 1 792 0 Ss+ ttyin 0xc535f410 getty
791 1 791 0 Ss+ ttyin 0xc538a010 getty
790 1 790 0 Ss+ ttyin 0xc5389c10 getty
789 1 789 0 Ss+ ttyin 0xc538ac10 getty
788 1 788 0 Ss+ ttyin 0xc538a810 getty
787 1 787 0 Ss+ ttyin 0xc5388c10 getty
786 1 786 0 Ss+ ttyin 0xc5388810 getty
785 1 785 0 Ss+ ttyin 0xc5389810 getty
764 1 764 0 Ss select 0xc0c03cbc inetd
737 1 737 0 Ss select 0xc0c03cbc moused
721 1 721 0 Ss nanslp 0xc0bb2724 cron
715 1 715 25 Ss pause 0xc52d7da8 sendmail
711 1 711 0 Ss select 0xc0c03cbc sendmail
705 1 705 0 Ss select 0xc0c03cbc sshd
587 1 587 0 Ss select 0xc0c03cbc syslogd
524 1 524 0 Ss select 0xc0c03cbc devd
328 1 328 65 Ss select 0xc0c03cbc dhclient
308 1 54 0 S+ select 0xc0c03cbc dhclient
53 0 0 0 SL sdflush 0xc0c0f9a4 [softdepflush]
52 0 0 0 SL syncer 0xc0bb254c [syncer]
51 0 0 0 SL vlruwt 0xc54f97f8 [vnlru]
50 0 0 0 SL psleep 0xc0c04164 [bufdaemon]
49 0 0 0 SL pgzero 0xc0c10560 [pagezero]
48 0 0 0 SL psleep 0xc0c10178 [vmdaemon]
47 0 0 0 SL psleep 0xc0c10140 [pagedaemon]
46 0 0 0 SL waiting_ 0xc0c05ecc [sctp_iterator]
45 0 0 0 WL [irq7: ppc0]
44 0 0 0 SL - 0xc5324e3c [fdc0]
43 0 0 0 WL [swi0: sio]
42 0 0 0 WL [irq12: psm0]
41 0 0 0 WL [irq1: atkbd0]
40 0 0 0 WL [irq15: ata1]
39 0 0 0 WL [irq14: ata0]
38 0 0 0 SL idle 0xc5326000 [aic_recovery1]
37 0 0 0 WL [irq22: ahd1]
36 0 0 0 SL idle 0xc52db000 [aic_recovery0]
35 0 0 0 SL aifthd 0xc52d7550 [aac0aif]
34 0 0 0 WL [irq18: aac0]
33 0 0 0 LL *rtentry 0xc50f2a00 [irq21: bge0]
32 0 0 0 SL usbevt 0xc520ba10 [usb2]
31 0 0 0 WL [irq23: ehci0]
30 0 0 0 SL usbevt 0xc5243210 [usb1]
29 0 0 0 WL [irq19: uhci1 ahd0]
28 0 0 0 SL usbtsk 0xc0bafe54 [usbtask-dr]
27 0 0 0 SL usbtsk 0xc0bafe40 [usbtask-hc]
26 0 0 0 SL usbevt 0xc5249210 [usb0]
25 0 0 0 WL [irq16: uhci0]
24 0 0 0 WL [irq9: acpi0]
23 0 0 0 SL - 0xc5213600 [kqueue taskq]
22 0 0 0 WL [swi6: task queue]
21 0 0 0 WL [swi6: Giant taskq]
9 0 0 0 SL - 0xc5213880 [thread taskq]
8 0 0 0 SL - 0xc5213900 [acpi_task_2]
7 0 0 0 SL - 0xc5213900 [acpi_task_1]
6 0 0 0 SL - 0xc5213900 [acpi_task_0]
20 0 0 0 WL [swi5: +]
19 0 0 0 WL [swi2: cambio]
5 0 0 0 SL ccb_scan 0xc0b81014 [xpt_thrd]
18 0 0 0 SL - 0xc0bb2554 [yarrow]
4 0 0 0 SL - 0xc0bb05cc [g_down]
3 0 0 0 SL - 0xc0bb05c8 [g_up]
2 0 0 0 SL - 0xc0bb05c0 [g_event]
17 0 0 0 WL [swi3: vm]
16 0 0 0 LL *inp 0xc51165f0 [swi4: clock sio]
15 0 0 0 WL [swi1: net]
14 0 0 0 RL [idle: cpu0]
13 0 0 0 RL [idle: cpu1]
12 0 0 0 RL CPU 2 [idle: cpu2]
11 0 0 0 RL CPU 3 [idle: cpu3]
1 0 1 0 SLs wait 0xc5114d48 [init]
10 0 0 0 SL audit_wo 0xc0c0f414 [audit]
0 0 0 0 WLs [swapper]
db> show pcpu
cpuid = 1
curthread = 0xc5117660: pid 16 "swi4: clock sio"
curpcb = 0xe3cb7d90
fpcurthread = none
idlethread = 0xc5115aa0: pid 13 "idle: cpu1"
APIC ID = 1
currentldt = 0x50
spin locks held:
db> show allpcpu
Current CPU: 1
cpuid = 0
curthread = 0xc5119880: pid 33 "irq21: bge0"
curpcb = 0xe3ccfd90
fpcurthread = none
idlethread = 0xc5115cc0: pid 14 "idle: cpu0"
APIC ID = 0
currentldt = 0x50
spin locks held:
cpuid = 1
curthread = 0xc5117660: pid 16 "swi4: clock sio"
curpcb = 0xe3cb7d90
fpcurthread = none
idlethread = 0xc5115aa0: pid 13 "idle: cpu1"
APIC ID = 1
currentldt = 0x50
spin locks held:
cpuid = 2
curthread = 0xc5115880: pid 12 "idle: cpu2"
curpcb = 0xe3ca5d90
fpcurthread = none
idlethread = 0xc5115880: pid 12 "idle: cpu2"
APIC ID = 6
currentldt = 0x50
spin locks held:
cpuid = 3
curthread = 0xc5115660: pid 11 "idle: cpu3"
curpcb = 0xe3ca2d90
fpcurthread = none
idlethread = 0xc5115660: pid 11 "idle: cpu3"
APIC ID = 7
currentldt = 0x50
spin locks held:
db> show locks
exclusive sleep mutex tcp r = 0 (0xc0c0664c) locked @
/usr/src/sys/netinet/tcp_timer.c:423
db> show alllocks
Process 22686 (sed) thread 0xc5c99440 (100188)
exclusive sleep mutex sleep mtxpool r = 0 (0xc50f1088) locked @
/usr/src/sys/kern/kern_descrip.c:2139
Process 21991 (sshd) thread 0xc57c6440 (100112)
exclusive sleep mutex rtentry r = 0 (0xc5874678) locked @
/usr/src/sys/net/route.c:1287
exclusive sleep mutex inp (tcpinp) r = 0 (0xc58a17c8) locked @
/usr/src/sys/netinet/tcp_usrreq.c:779
exclusive sx so_snd_sx r = 0 (0xc66cb748) locked @
/usr/src/sys/kern/uipc_sockbuf.c:145
Process 21982 (sshd) thread 0xc5a3bcc0 (100151)
exclusive sx so_rcv_sx r = 0 (0xc7600218) locked @
/usr/src/sys/kern/uipc_sockbuf.c:145
Process 83121 (sshd) thread 0xc5c1e000 (100154)
exclusive sx so_rcv_sx r = 0 (0xc57126bc) locked @
/usr/src/sys/kern/uipc_sockbuf.c:145
Process 801 (sshd) thread 0xc5a39aa0 (100127)
exclusive sx so_rcv_sx r = 0 (0xc57129d4) locked @
/usr/src/sys/kern/uipc_sockbuf.c:145
Process 33 (irq21: bge0) thread 0xc5119880 (100025)
exclusive sleep mutex rtentry r = 0 (0xc5719d80) locked @
/usr/src/sys/net/route.c:197
Process 16 (swi4: clock sio) thread 0xc5117660 (100014)
exclusive sleep mutex tcp r = 0 (0xc0c0664c) locked @
/usr/src/sys/netinet/tcp_timer.c:423
db> show lockedvnods
Locked vnodes
db> alltrace
Tracing command sed pid 22686 tid 100188 td 0xc5c99440
sched_switch(c5c99440,0,6,17b,3bcb1dac,...) at sched_switch+0x49b
mi_switch(6,0,c0a9e6f6,b6,c5c99440,...) at mi_switch+0x217
critical_exit(c0bba580,ebec8adc,c074280d,c0bba580,1,...) at
critical_exit+0x99
spinlock_exit(c0bba580,1,c0a9b5fa,f4) at spinlock_exit+0x10
_mtx_unlock_spin_flags(c0bba580,0,c0a99497,2cc,c09f2677,...) at
_mtx_unlock_spin_flags+0xfd
intr_event_schedule_thread(c515e500,0,5,c5c99440,c515e500,...) at
intr_event_schedule_thread+0x114
intr_execute_handlers(c5107f34,ebec8b58,ebec8bb4,c09ec804,34,...) at
intr_execute_handlers+0x12b
lapic_handle_intr(34,ebec8b58) at lapic_handle_intr+0x3f
Xapic_isr1() at Xapic_isr1+0x34
--- interrupt, eip = 0xc0742aa2, esp = 0xebec8b98, ebp = 0xebec8bb4 ---
_mtx_unlock_flags(c50f1088,0,c0a9881a,86c,fc000,...) at
_mtx_unlock_flags+0x62
fdrop(c5c70e58,c5c99440,fc000,5,7,...) at fdrop+0x93
mmap(c5c99440,ebec8cfc,1c,c0a5612b,c0b4beb8,...) at mmap+0x3a7
syscall(ebec8d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (477, FreeBSD ELF32, mmap), eip = 0x28057fe3, esp =
0xbfbfe00c, ebp = 0xbfbfe048 ---
Tracing command sh pid 22684 tid 100250 td 0xca6f1aa0
sched_switch(ca6f1aa0,0,1,180,3b60a3d8,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(ca6f1aa0,0,c0aa02cc,19b,cb49eaa0,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba600,1,c0a9b5fa,5c,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(cb49eaa0,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(cb49eaa0,cb49eb30,15c,c0aa2401,0,...) at _sleep+0x359
kern_wait(ca6f1aa0,ffffffff,ebfbdc2c,2,0,...) at kern_wait+0xa66
wait4(ca6f1aa0,ebfbdcfc,10,c0aa2214,c0b492a8,...) at wait4+0x3b
syscall(ebfbdd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (7, FreeBSD ELF32, wait4), eip = 0x28149e7b, esp =
0xbfbfcd3c, ebp = 0xbfbfcd58 ---
Tracing command cat pid 22683 tid 100077 td 0xc55d5440
sched_switch(c55d5440,0,1,180,3bd01c1c,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c55d5440,0,c0aa02cc,19b,c5542c38,...) at sleepq_switch+0xf0
sleepq_catch_signals(ebcb1a70,c074280d,c0bba580,1,5a,...) at
sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c5542c38,0,c0a9d1f9,da,0,...) at
sleepq_timedwait_sig+0x17
_sleep(c5542c38,0,15a,c0aa3ca0,493e0,...) at _sleep+0x331
ttysleep(c5542c00,c5542c38,15a,c0aa3ca0,493e0,...) at ttysleep+0x39
ttwrite(c5542c00,ebcb1c60,0,ebcb1bbc,c071e768,...) at ttwrite+0x5aa
ptswrite(c6b90c00,ebcb1c60,0,174,1000,...) at ptswrite+0x38
giant_write(c6b90c00,ebcb1c60,0,0,c0b562e0,...) at giant_write+0x48
devfs_write_f(c5ec4c18,ebcb1c60,c6abe900,0,c55d5440,...) at
devfs_write_f+0x98
dofilewrite(ebcb1c60,ffffffff,ffffffff,0,c5ec4c18,...) at dofilewrite+0x95
kern_writev(c55d5440,1,ebcb1c60,282012bc,d44,...) at kern_writev+0x58
write(c55d5440,ebcb1cfc,c,c0a5612b,c0b49260,...) at write+0x4f
syscall(ebcb1d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (4, FreeBSD ELF32, write), eip = 0x28154483, esp =
0xbfbfeb9c, ebp = 0xbfbfec48 ---
Tracing command sh pid 22651 tid 100202 td 0xc6eedcc0
sched_switch(c6eedcc0,0,1,180,3afa7a8c,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c6eedcc0,0,c0aa02cc,19b,c5bfb318,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba600,1,c0a9b5fa,4c,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c5bfb318,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(c5bfb318,c5bfb488,14c,c0aa2013,0,...) at _sleep+0x359
pipe_read(c58af0d8,ebf31c60,c587e700,0,c6eedcc0,...) at pipe_read+0x3f6
dofileread(ebf31c60,ffffffff,ffffffff,0,c58af0d8,...) at dofileread+0x96
kern_readv(c6eedcc0,3,ebf31c60,bfbfcf1c,80,...) at kern_readv+0x58
read(c6eedcc0,ebf31cfc,c,ebf31d38,c0b49248,...) at read+0x4f
syscall(ebf31d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x281c54a3, esp =
0xbfbfce6c, ebp = 0xbfbfcfb8 ---
Tracing command csh pid 22205 tid 100212 td 0xc6eec660
sched_switch(c6eec660,0,1,180,3a7253f8,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c6eec660,0,c0aa02cc,19b,c6ef05b0,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba580,1,c0a9b5fa,68,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c6ef05b0,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(c6ef05b0,c6ef05e0,168,c0a5f873,0,...) at _sleep+0x359
kern_sigsuspend(c6eec660,0,0,0,0,...) at kern_sigsuspend+0xe4
sigsuspend(c6eec660,ebf13cfc,4,c0aa22db,c0b4b1f8,...) at sigsuspend+0x4d
syscall(ebf13d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (341, FreeBSD ELF32, sigsuspend), eip = 0x28166ffb, esp =
0xbfbfe73c, ebp = 0xbfbfe7a8 ---
Tracing command su pid 22198 tid 100200 td 0xc55d2000
sched_switch(c55d2000,0,1,180,367f1a58,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c55d2000,0,c0aa02cc,19b,c6eeb550,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba580,1,c0a9b5fa,5c,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c6eeb550,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(c6eeb550,c6eeb5e0,15c,c0aa2401,0,...) at _sleep+0x359
kern_wait(c55d2000,56bd,ebc96c2c,2,0,...) at kern_wait+0xa66
wait4(c55d2000,ebc96cfc,10,ebc96d38,c0b492a8,...) at wait4+0x3b
syscall(ebc96d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (7, FreeBSD ELF32, wait4), eip = 0x2810ae7b, esp =
0xbfbfe79c, ebp = 0xbfbfe7b8 ---
Tracing command csh pid 21992 tid 100196 td 0xc5c98220
sched_switch(c5c98220,0,1,180,2193d730,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5c98220,0,c0aa02cc,19b,c5c9bb00,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba580,1,c0a9b5fa,68,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c5c9bb00,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(c5c9bb00,c5c9bb30,168,c0a5f873,0,...) at _sleep+0x359
kern_sigsuspend(c5c98220,0,0,0,0,...) at kern_sigsuspend+0xe4
sigsuspend(c5c98220,ebeb0cfc,4,c0aa22db,c0b4b1f8,...) at sigsuspend+0x4d
syscall(ebeb0d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (341, FreeBSD ELF32, sigsuspend), eip = 0x28166ffb, esp =
0xbfbfe80c, ebp = 0xbfbfe878 ---
Tracing command sshd pid 21991 tid 100112 td 0xc57c6440
sched_switch(c57c6440,0,1,17b,917954c4,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0a32,2e2,c57c8a50,...) at mi_switch+0x217
turnstile_wait(c57c8a50,c5119880,0,17a,c5719d80,...) at turnstile_wait+0x4cb
_mtx_lock_sleep(c5719d80,c57c6440,0,c0aaa355,517,...) at
_mtx_lock_sleep+0x15a
_mtx_lock_flags(c5719d80,0,c0aaa355,517,ebd27964,...) at
_mtx_lock_flags+0xef
rt_check(ebd2795c,ebd27978,cb4ee2d0,c0b4f564,c0aaa355,...) at rt_check+0x113
arpresolve(c526a400,c5874618,c550fb00,cb4ee2d0,ebd27992,...) at
arpresolve+0xb0
ether_output(c526a400,c550fb00,cb4ee2d0,c5874618,c57c6440,...) at
ether_output+0x7e
ip_output(c550fb00,0,ebd27a00,0,0,...) at ip_output+0xa04
tcp_output(c58a4378,c6148500,c0ab354d,30b,c58a17c8,...) at tcp_output+0x13f0
tcp_usr_send(c66cb630,0,c6148500,0,0,...) at tcp_usr_send+0x25a
sosend_generic(c66cb630,0,ebd27c60,0,0,...) at sosend_generic+0x50d
sosend(c66cb630,0,ebd27c60,0,0,...) at sosend+0x54
soo_write(c5be9d80,ebd27c60,c605ad00,0,c57c6440,...) at soo_write+0x4b
dofilewrite(ebd27c60,ffffffff,ffffffff,0,c5be9d80,...) at dofilewrite+0x95
kern_writev(c57c6440,3,ebd27c60,28535550,0,...) at kern_writev+0x58
write(c57c6440,ebd27cfc,c,c0a5612b,c0b49260,...) at write+0x4f
syscall(ebd27d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (4, FreeBSD ELF32, write), eip = 0x283e3483, esp =
0xbfbfe5cc, ebp = 0xbfbfe5e8 ---
Tracing command sshd pid 21982 tid 100151 td 0xc5a3bcc0
sched_switch(c5a3bcc0,0,1,180,f317eb00,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5a3bcc0,0,c0aa02cc,19b,c7600248,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba600,1,c0a9b5fa,58,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c7600248,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(c7600248,c7600200,158,c0aa509f,0) at _sleep+0x359
sbwait(c76001dc,1,c0aa51b4,5ae,c7600200,...) at sbwait+0x76
soreceive_generic(c760018c,0,ebdf5c60,0,0,...) at soreceive_generic+0x42f
soreceive(c760018c,0,ebdf5c60,0,0,...) at soreceive+0x4d
soo_read(c6ee14c8,ebdf5c60,c99eed00,0,c5a3bcc0,...) at soo_read+0x3b
dofileread(ebdf5c60,ffffffff,ffffffff,0,c6ee14c8,...) at dofileread+0x96
kern_readv(c5a3bcc0,5,ebdf5c60,bfbfe64c,4,...) at kern_readv+0x58
read(c5a3bcc0,ebdf5cfc,c,c0ab76e1,c0b49248,...) at read+0x4f
syscall(ebdf5d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x283e34a3, esp =
0xbfbfe60c, ebp = 0xbfbfe628 ---
Tracing command gmake pid 20101 tid 100252 td 0xca6f1660
sched_switch(ca6f1660,0,1,180,2da8e9cc,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(ca6f1660,0,c0aa02cc,19b,cb4a0000,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba600,1,c0a9b5fa,5c,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(cb4a0000,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(cb4a0000,cb4a0090,15c,c0aa2401,0,...) at _sleep+0x359
kern_wait(ca6f1660,ffffffff,ebfb7c2c,0,0,...) at kern_wait+0xa66
wait4(ca6f1660,ebfb7cfc,10,c0aa22db,c0b492a8,...) at wait4+0x3b
syscall(ebfb7d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (7, FreeBSD ELF32, wait4), eip = 0x281f9e7b, esp =
0xbfbfd49c, ebp = 0xbfbfd4b8 ---
Tracing command sh pid 20097 tid 100201 td 0xc6eee000
sched_switch(c6eee000,0,1,180,cca81b5c,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c6eee000,0,c0aa02cc,19b,c6eeb7f8,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba600,1,c0a9b5fa,5c,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c6eeb7f8,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(c6eeb7f8,c6eeb888,15c,c0aa2401,0,...) at _sleep+0x359
kern_wait(c6eee000,ffffffff,ebf34c2c,2,0,...) at kern_wait+0xa66
wait4(c6eee000,ebf34cfc,10,c0aa23e0,c0b492a8,...) at wait4+0x3b
syscall(ebf34d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (7, FreeBSD ELF32, wait4), eip = 0x28149e7b, esp =
0xbfbfdf3c, ebp = 0xbfbfdf58 ---
Tracing command gmake pid 20096 tid 100064 td 0xc55a1440
sched_switch(c55a1440,0,1,180,cb25aefa,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c55a1440,0,c0aa02cc,19b,c559daa0,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba580,1,c0a9b5fa,5c,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c559daa0,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(c559daa0,c559db30,15c,c0aa2401,0,...) at _sleep+0x359
kern_wait(c55a1440,ffffffff,ebc76c2c,0,0,...) at kern_wait+0xa66
wait4(c55a1440,ebc76cfc,10,c0aa22db,c0b492a8,...) at wait4+0x3b
syscall(ebc76d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (7, FreeBSD ELF32, wait4), eip = 0x281f9e7b, esp =
0xbfbfd58c, ebp = 0xbfbfd5a8 ---
Tracing command sh pid 84323 tid 100257 td 0xca6edaa0
sched_switch(ca6edaa0,0,1,180,c8f4c658,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(ca6edaa0,0,c0aa02cc,19b,cb4a0d48,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba600,1,c0a9b5fa,5c,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(cb4a0d48,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(cb4a0d48,cb4a0dd8,15c,c0aa2401,0,...) at _sleep+0x359
kern_wait(ca6edaa0,ffffffff,ebfa8c2c,2,0,...) at kern_wait+0xa66
wait4(ca6edaa0,ebfa8cfc,10,c0aa23e0,c0b492a8,...) at wait4+0x3b
syscall(ebfa8d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (7, FreeBSD ELF32, wait4), eip = 0x28149e7b, esp =
0xbfbfdf1c, ebp = 0xbfbfdf38 ---
Tracing command gmake pid 84322 tid 100235 td 0xca6ecaa0
sched_switch(ca6ecaa0,0,1,180,1a565222,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(ca6ecaa0,0,c0aa02cc,19b,ca6ef2a8,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba600,1,c0a9b5fa,5c,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(ca6ef2a8,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(ca6ef2a8,ca6ef338,15c,c0aa2401,0,...) at _sleep+0x359
kern_wait(ca6ecaa0,ffffffff,ebf93c2c,0,0,...) at kern_wait+0xa66
wait4(ca6ecaa0,ebf93cfc,10,c0aa22db,c0b492a8,...) at wait4+0x3b
syscall(ebf93d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (7, FreeBSD ELF32, wait4), eip = 0x281f9e7b, esp =
0xbfbfd59c, ebp = 0xbfbfd5b8 ---
Tracing command sh pid 84318 tid 100167 td 0xc5c1c220
sched_switch(c5c1c220,0,1,180,18ffc18c,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5c1c220,0,c0aa02cc,19b,c5c20d48,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba600,1,c0a9b5fa,5c,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c5c20d48,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(c5c20d48,c5c20dd8,15c,c0aa2401,0,...) at _sleep+0x359
kern_wait(c5c1c220,ffffffff,ebe25c2c,2,0,...) at kern_wait+0xa66
wait4(c5c1c220,ebe25cfc,10,c0aa23e0,c0b492a8,...) at wait4+0x3b
syscall(ebe25d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (7, FreeBSD ELF32, wait4), eip = 0x28149e7b, esp =
0xbfbfdecc, ebp = 0xbfbfdee8 ---
Tracing command gmake pid 84317 tid 100208 td 0xc6eed000
sched_switch(c6eed000,0,1,180,177a6518,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c6eed000,0,c0aa02cc,19b,c6eefaa0,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba600,1,c0a9b5fa,5c,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c6eefaa0,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(c6eefaa0,c6eefb30,15c,c0aa2401,0,...) at _sleep+0x359
kern_wait(c6eed000,ffffffff,ebf1fc2c,0,0,...) at kern_wait+0xa66
wait4(c6eed000,ebf1fcfc,10,c0aa22db,c0b492a8,...) at wait4+0x3b
syscall(ebf1fd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (7, FreeBSD ELF32, wait4), eip = 0x281f9e7b, esp =
0xbfbfd68c, ebp = 0xbfbfd6a8 ---
Tracing command gmake pid 84316 tid 100143 td 0xc5bbf000
sched_switch(c5bbf000,0,1,180,15579c9a,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5bbf000,0,c0aa02cc,19b,c5bc1d48,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba580,1,c0a9b5fa,5c,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c5bc1d48,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(c5bc1d48,c5bc1dd8,15c,c0aa2401,0,...) at _sleep+0x359
kern_wait(c5bbf000,ffffffff,ebe0dc2c,0,0,...) at kern_wait+0xa66
wait4(c5bbf000,ebe0dcfc,10,c0aa22db,c0b492a8,...) at wait4+0x3b
syscall(ebe0dd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (7, FreeBSD ELF32, wait4), eip = 0x281f9e7b, esp =
0xbfbfd76c, ebp = 0xbfbfd788 ---
Tracing command sh pid 84315 tid 100224 td 0xc7220880
sched_switch(c7220880,0,1,180,11e9dfac,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c7220880,0,c0aa02cc,19b,c7224550,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba600,1,c0a9b5fa,5c,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c7224550,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(c7224550,c72245e0,15c,c0aa2401,0,...) at _sleep+0x359
kern_wait(c7220880,ffffffff,ebf66c2c,2,0,...) at kern_wait+0xa66
wait4(c7220880,ebf66cfc,10,ebf66d38,c0b492a8,...) at wait4+0x3b
syscall(ebf66d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (7, FreeBSD ELF32, wait4), eip = 0x28149e7b, esp =
0xbfbfe41c, ebp = 0xbfbfe438 ---
Tracing command make pid 20720 tid 100096 td 0xc562b000
sched_switch(c562b000,0,1,180,115fc284,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c562b000,0,c0aa02cc,19b,c5633000,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba580,1,c0a9b5fa,5c,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c5633000,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(c5633000,c5633090,15c,c0aa2401,0,...) at _sleep+0x359
kern_wait(c562b000,ffffffff,ebce6c2c,0,0,...) at kern_wait+0xa66
wait4(c562b000,ebce6cfc,10,c0aa235c,c0b492a8,...) at wait4+0x3b
syscall(ebce6d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (7, FreeBSD ELF32, wait4), eip = 0x80684cf, esp =
0xbfbfdc9c, ebp = 0xbfbfdcb8 ---
Tracing command sh pid 20719 tid 100182 td 0xc5c1e880
sched_switch(c5c1e880,0,1,180,6a097586,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5c1e880,0,c0aa02cc,19b,c5c97550,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba580,1,c0a9b5fa,5c,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c5c97550,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(c5c97550,c5c975e0,15c,c0aa2401,0,...) at _sleep+0x359
kern_wait(c5c1e880,ffffffff,ebe58c2c,2,0,...) at kern_wait+0xa66
wait4(c5c1e880,ebe58cfc,10,ebe58d38,c0b492a8,...) at wait4+0x3b
syscall(ebe58d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (7, FreeBSD ELF32, wait4), eip = 0x28149e7b, esp =
0xbfbfe7fc, ebp = 0xbfbfe818 ---
Tracing command make pid 20703 tid 100178 td 0xc5c85220
sched_switch(c5c85220,0,1,180,696d66ec,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5c85220,0,c0aa02cc,19b,c5c87aa0,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba600,1,c0a9b5fa,5c,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c5c87aa0,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(c5c87aa0,c5c87b30,15c,c0aa2401,0,...) at _sleep+0x359
kern_wait(c5c85220,ffffffff,ebe86c2c,0,0,...) at kern_wait+0xa66
wait4(c5c85220,ebe86cfc,10,c0aa235c,c0b492a8,...) at wait4+0x3b
syscall(ebe86d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (7, FreeBSD ELF32, wait4), eip = 0x80684cf, esp =
0xbfbfdd9c, ebp = 0xbfbfddb8 ---
Tracing command csh pid 86175 tid 100093 td 0xc562b660
sched_switch(c562b660,0,1,180,ffd380c4,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c562b660,0,c0aa02cc,19b,c5528410,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba580,1,c0a9b5fa,59,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c5528410,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(c5528410,0,159,c0aa3c6f,0,...) at _sleep+0x359
ttysleep(c5528400,c5528410,159,c0aa3c6f,0,...) at ttysleep+0x39
ttread(c5528400,ebcefc60,0,c843fd00,ebcefc60,...) at ttread+0x4e7
ptsread(c843fd00,ebcefc60,0,168,0,...) at ptsread+0x3e
giant_read(c843fd00,ebcefc60,0,0,1,...) at giant_read+0x48
devfs_read_f(c6ee1ca8,ebcefc60,c83ea300,0,c562b660,...) at devfs_read_f+0x71
dofileread(ebcefc60,ffffffff,ffffffff,0,c6ee1ca8,...) at dofileread+0x96
kern_readv(c562b660,10,ebcefc60,bfbfe836,1,...) at kern_readv+0x58
read(c562b660,ebcefcfc,c,c0a8d525,c0b49248,...) at read+0x4f
syscall(ebcefd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x281f84a3, esp =
0xbfbfe7ec, ebp = 0xbfbfe808 ---
Tracing command su pid 85857 tid 100111 td 0xc57c6660
sched_switch(c57c6660,0,1,180,35f498b4,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c57c6660,0,c0aa02cc,19b,c57ca7f8,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba580,1,c0a9b5fa,5c,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c57ca7f8,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(c57ca7f8,c57ca888,15c,c0aa2401,0,...) at _sleep+0x359
kern_wait(c57c6660,1509f,ebd2ac2c,2,0,...) at kern_wait+0xa66
wait4(c57c6660,ebd2acfc,10,ebd2ad38,c0b492a8,...) at wait4+0x3b
syscall(ebd2ad38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (7, FreeBSD ELF32, wait4), eip = 0x2810ae7b, esp =
0xbfbfe79c, ebp = 0xbfbfe7b8 ---
Tracing command csh pid 84932 tid 100073 td 0xc55d5cc0
sched_switch(c55d5cc0,0,1,180,3faccc7c,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c55d5cc0,0,c0aa02cc,19b,c55d1308,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba600,1,c0a9b5fa,68,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c55d1308,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(c55d1308,c55d1338,168,c0a5f873,0,...) at _sleep+0x359
kern_sigsuspend(c55d5cc0,0,0,0,0,...) at kern_sigsuspend+0xe4
sigsuspend(c55d5cc0,ebcbdcfc,4,c0aa22db,c0b4b1f8,...) at sigsuspend+0x4d
syscall(ebcbdd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (341, FreeBSD ELF32, sigsuspend), eip = 0x28166ffb, esp =
0xbfbfe80c, ebp = 0xbfbfe878 ---
Tracing command sshd pid 84911 tid 100068 td 0xc559eaa0
sched_switch(c559eaa0,0,1,180,7727e34,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c559eaa0,0,c0aa02cc,19b,c0c03cbc,...) at sleepq_switch+0xf0
sleepq_catch_signals(c0742baa,c0c03ca4,0,c0a9b5fa,c559eaa0,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c0c03cbc,c0c03ca4,c0ab9fa0,101,0,...) at
sleepq_wait_sig+0x14
_cv_wait_sig(c0c03cbc,c0c03ca4,c0aa1e52,30f,c0a966e4,...) at
_cv_wait_sig+0x1fc
kern_select(c559eaa0,9,285030e4,285030e8,0,0,3fd0,285030e4) at
kern_select+0x6cc
select(c559eaa0,ebc6acfc,14,c0a5612b,c0b49ab8,...) at select+0x5e
syscall(ebc6ad38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x283e3423, esp =
0xbfbfe5fc, ebp = 0xbfbfe648 ---
Tracing command sshd pid 83121 tid 100154 td 0xc5c1e000
sched_switch(c5c1e000,0,1,180,6e89b21c,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5c1e000,0,c0aa02cc,19b,c57126ec,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba600,1,c0a9b5fa,58,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c57126ec,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(c57126ec,c57126a4,158,c0aa509f,0) at _sleep+0x359
sbwait(c5712680,1,c0aa51b4,5ae,c57126a4,...) at sbwait+0x76
soreceive_generic(c5712630,0,ebe4cc60,0,0,...) at soreceive_generic+0x42f
soreceive(c5712630,0,ebe4cc60,0,0,...) at soreceive+0x4d
soo_read(c724d360,ebe4cc60,c8467e00,0,c5c1e000,...) at soo_read+0x3b
dofileread(ebe4cc60,ffffffff,ffffffff,0,c724d360,...) at dofileread+0x96
kern_readv(c5c1e000,5,ebe4cc60,bfbfe64c,4,...) at kern_readv+0x58
read(c5c1e000,ebe4ccfc,c,c0ab76e1,c0b49248,...) at read+0x4f
syscall(ebe4cd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x283e34a3, esp =
0xbfbfe60c, ebp = 0xbfbfe628 ---
Tracing command sh pid 25073 tid 100185 td 0xc5c99aa0
sched_switch(c5c99aa0,0,1,180,55885b24,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5c99aa0,0,c0aa02cc,19b,c5c97d48,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba580,1,c0a9b5fa,5c,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c5c97d48,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(c5c97d48,c5c97dd8,15c,c0aa2401,0,...) at _sleep+0x359
kern_wait(c5c99aa0,ffffffff,ebed1c2c,2,0,...) at kern_wait+0xa66
wait4(c5c99aa0,ebed1cfc,10,c0aa23e0,c0b492a8,...) at wait4+0x3b
syscall(ebed1d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (7, FreeBSD ELF32, wait4), eip = 0x28149e7b, esp =
0xbfbfdb5c, ebp = 0xbfbfdb78 ---
Tracing command make pid 7399 tid 100088 td 0xc562c220
sched_switch(c562c220,0,1,180,f024bcc0,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c562c220,0,c0aa02cc,19b,c562aaa0,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba600,1,c0a9b5fa,5c,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c562aaa0,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(c562aaa0,c562ab30,15c,c0aa2401,0,...) at _sleep+0x359
kern_wait(c562c220,ffffffff,ebcfec2c,0,0,...) at kern_wait+0xa66
wait4(c562c220,ebcfecfc,10,c0aa235c,c0b492a8,...) at wait4+0x3b
syscall(ebcfed38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (7, FreeBSD ELF32, wait4), eip = 0x80684cf, esp =
0xbfbfde4c, ebp = 0xbfbfde68 ---
Tracing command csh pid 810 tid 100097 td 0xc55d6cc0
sched_switch(c55d6cc0,0,1,180,11513c44,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c55d6cc0,0,c0aa02cc,19b,c5633308,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba600,1,c0a9b5fa,68,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c5633308,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(c5633308,c5633338,168,c0a5f873,0,...) at _sleep+0x359
kern_sigsuspend(c55d6cc0,0,0,0,0,...) at kern_sigsuspend+0xe4
sigsuspend(c55d6cc0,ebcd2cfc,4,c0aa22db,c0b4b1f8,...) at sigsuspend+0x4d
syscall(ebcd2d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (341, FreeBSD ELF32, sigsuspend), eip = 0x28166ffb, esp =
0xbfbfe72c, ebp = 0xbfbfe798 ---
Tracing command su pid 809 tid 100122 td 0xc5a3b660
sched_switch(c5a3b660,0,1,180,5f8c36e4,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5a3b660,0,c0aa02cc,19b,c5a38550,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba600,1,c0a9b5fa,5c,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c5a38550,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(c5a38550,c5a385e0,15c,c0aa2401,0,...) at _sleep+0x359
kern_wait(c5a3b660,32a,ebdecc2c,2,0,...) at kern_wait+0xa66
wait4(c5a3b660,ebdeccfc,10,ebdecd38,c0b492a8,...) at wait4+0x3b
syscall(ebdecd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (7, FreeBSD ELF32, wait4), eip = 0x2810ae7b, esp =
0xbfbfe78c, ebp = 0xbfbfe7a8 ---
Tracing command csh pid 805 tid 100126 td 0xc5a39cc0
sched_switch(c5a39cc0,0,1,180,8e4b9782,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5a39cc0,0,c0aa02cc,19b,c5a3c060,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba580,1,c0a9b5fa,68,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c5a3c060,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(c5a3c060,c5a3c090,168,c0a5f873,0,...) at _sleep+0x359
kern_sigsuspend(c5a39cc0,0,0,0,0,...) at kern_sigsuspend+0xe4
sigsuspend(c5a39cc0,ebde0cfc,4,c0aa22db,c0b4b1f8,...) at sigsuspend+0x4d
syscall(ebde0d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (341, FreeBSD ELF32, sigsuspend), eip = 0x28166ffb, esp =
0xbfbfe80c, ebp = 0xbfbfe878 ---
Tracing command sshd pid 804 tid 100070 td 0xc559e660
sched_switch(c559e660,0,1,180,2d9d0594,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c559e660,0,c0aa02cc,19b,c0c03cbc,...) at sleepq_switch+0xf0
sleepq_catch_signals(c0742baa,c0c03ca4,0,c0a9b5fa,c559e660,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c0c03cbc,c0c03ca4,c0ab9fa0,101,0,...) at
sleepq_wait_sig+0x14
_cv_wait_sig(c0c03cbc,c0c03ca4,c0aa1e52,30f,c0ab354d,...) at
_cv_wait_sig+0x1fc
kern_select(c559e660,9,285030e4,285030e8,0,0,0,280e05e8) at
kern_select+0x6cc
select(c559e660,ebc64cfc,14,c0a8d525,c0b49ab8,...) at select+0x5e
syscall(ebc64d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x283e3423, esp =
0xbfbfe5fc, ebp = 0xbfbfe648 ---
Tracing command sshd pid 801 tid 100127 td 0xc5a39aa0
sched_switch(c5a39aa0,0,1,180,f0ecc0ae,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5a39aa0,0,c0aa02cc,19b,c5712a04,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba600,1,c0a9b5fa,58,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c5712a04,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(c5712a04,c57129bc,158,c0aa509f,0) at _sleep+0x359
sbwait(c5712998,1,c0aa51b4,5ae,c57129bc,...) at sbwait+0x76
soreceive_generic(c5712948,0,ebdddc60,0,0,...) at soreceive_generic+0x42f
soreceive(c5712948,0,ebdddc60,0,0,...) at soreceive+0x4d
soo_read(c55cbd80,ebdddc60,c5630000,0,c5a39aa0,...) at soo_read+0x3b
dofileread(ebdddc60,ffffffff,ffffffff,0,c55cbd80,...) at dofileread+0x96
kern_readv(c5a39aa0,5,ebdddc60,bfbfe64c,4,...) at kern_readv+0x58
read(c5a39aa0,ebdddcfc,c,c0ab76e1,c0b49248,...) at read+0x4f
syscall(ebdddd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x283e34a3, esp =
0xbfbfe60c, ebp = 0xbfbfe628 ---
Tracing command login pid 793 tid 100082 td 0xc55d2880
sched_switch(c55d2880,0,1,180,7a9d1d84,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c55d2880,0,c0aa02cc,19b,c535d810,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba600,1,c0a9b5fa,59,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c535d810,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(c535d810,0,159,c0aa3c6f,0,...) at _sleep+0x359
ttysleep(c535d800,c535d810,159,c0aa3c6f,0,...) at ttysleep+0x39
ttread(c535d800,ebca2c60,0,ebca2bb8,c071e7e8,...) at ttread+0x4e7
ttyread(c52d8300,ebca2c60,0,168,0,...) at ttyread+0x39
giant_read(c52d8300,ebca2c60,0,0,1,...) at giant_read+0x48
devfs_read_f(c5634ca8,ebca2c60,c511c800,0,c55d2880,...) at devfs_read_f+0x71
dofileread(ebca2c60,ffffffff,ffffffff,0,c5634ca8,...) at dofileread+0x96
kern_readv(c55d2880,0,ebca2c60,bfbfe9bb,1,...) at kern_readv+0x58
read(c55d2880,ebca2cfc,c,c0aa23f0,c0b49248,...) at read+0x4f
syscall(ebca2d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x2817f4a3, esp =
0xbfbfe72c, ebp = 0xbfbfe9c8 ---
Tracing command getty pid 792 tid 100069 td 0xc559e880
sched_switch(c559e880,0,1,180,397a7f30,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c559e880,0,c0aa02cc,19b,c535f410,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba600,1,c0a9b5fa,59,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c535f410,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(c535f410,0,159,c0aa3c6f,0,...) at _sleep+0x359
ttysleep(c535f400,c535f410,159,c0aa3c6f,0,...) at ttysleep+0x39
ttread(c535f400,ebc67c60,0,ebc67b9c,c0679bfd,...) at ttread+0x4e7
ttyread(c5380800,ebc67c60,0,ebc67bb8,c071e7e8,...) at ttyread+0x39
scread(c5380800,ebc67c60,0,168,0,...) at scread+0x2d
giant_read(c5380800,ebc67c60,0,0,1,...) at giant_read+0x48
devfs_read_f(c58739d8,ebc67c60,c511c800,0,c559e880,...) at devfs_read_f+0x71
dofileread(ebc67c60,ffffffff,ffffffff,0,c58739d8,...) at dofileread+0x96
kern_readv(c559e880,0,ebc67c60,bfbfee4b,1,...) at kern_readv+0x58
read(c559e880,ebc67cfc,c,c0ace8dc,c0b49248,...) at read+0x4f
syscall(ebc67d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x281644a3, esp =
0xbfbfee2c, ebp = 0xbfbfee58 ---
Tracing command getty pid 791 tid 100092 td 0xc562b880
sched_switch(c562b880,0,1,180,391d0a04,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c562b880,0,c0aa02cc,19b,c538a010,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba600,1,c0a9b5fa,59,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c538a010,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(c538a010,0,159,c0aa3c6f,0,...) at _sleep+0x359
ttysleep(c538a000,c538a010,159,c0aa3c6f,0,...) at ttysleep+0x39
ttread(c538a000,ebcf2c60,0,ebcf2b9c,c0679bfd,...) at ttread+0x4e7
ttyread(c5380700,ebcf2c60,0,ebcf2bb8,c071e7e8,...) at ttyread+0x39
scread(c5380700,ebcf2c60,0,168,0,...) at scread+0x2d
giant_read(c5380700,ebcf2c60,0,0,1,...) at giant_read+0x48
devfs_read_f(c55cbca8,ebcf2c60,c511c800,0,c562b880,...) at devfs_read_f+0x71
dofileread(ebcf2c60,ffffffff,ffffffff,0,c55cbca8,...) at dofileread+0x96
kern_readv(c562b880,0,ebcf2c60,bfbfee4b,1,...) at kern_readv+0x58
read(c562b880,ebcf2cfc,c,c0ace8dc,c0b49248,...) at read+0x4f
syscall(ebcf2d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x281644a3, esp =
0xbfbfee2c, ebp = 0xbfbfee58 ---
Tracing command getty pid 790 tid 100098 td 0xc55d6aa0
sched_switch(c55d6aa0,0,1,180,38e12210,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c55d6aa0,0,c0aa02cc,19b,c5389c10,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba580,1,c0a9b5fa,59,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c5389c10,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(c5389c10,0,159,c0aa3c6f,0,...) at _sleep+0x359
ttysleep(c5389c00,c5389c10,159,c0aa3c6f,0,...) at ttysleep+0x39
ttread(c5389c00,ebccfc60,0,ebccfb9c,c0679bfd,...) at ttread+0x4e7
ttyread(c5380600,ebccfc60,0,ebccfbb8,c071e7e8,...) at ttyread+0x39
scread(c5380600,ebccfc60,0,168,0,...) at scread+0x2d
giant_read(c5380600,ebccfc60,0,0,1,...) at giant_read+0x48
devfs_read_f(c55a6bd0,ebccfc60,c511c800,0,c55d6aa0,...) at devfs_read_f+0x71
dofileread(ebccfc60,ffffffff,ffffffff,0,c55a6bd0,...) at dofileread+0x96
kern_readv(c55d6aa0,0,ebccfc60,bfbfee4b,1,...) at kern_readv+0x58
read(c55d6aa0,ebccfcfc,c,c0ace8dc,c0b49248,...) at read+0x4f
syscall(ebccfd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x281644a3, esp =
0xbfbfee2c, ebp = 0xbfbfee58 ---
Tracing command getty pid 789 tid 100130 td 0xc5a39440
sched_switch(c5a39440,0,1,180,3940407c,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5a39440,0,c0aa02cc,19b,c538ac10,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba580,1,c0a9b5fa,59,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c538ac10,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(c538ac10,0,159,c0aa3c6f,0,...) at _sleep+0x359
ttysleep(c538ac00,c538ac10,159,c0aa3c6f,0,...) at ttysleep+0x39
ttread(c538ac00,ebdd4c60,0,ebdd4b9c,c0679bfd,...) at ttread+0x4e7
ttyread(c5380500,ebdd4c60,0,ebdd4bb8,c071e7e8,...) at ttyread+0x39
scread(c5380500,ebdd4c60,0,168,0,...) at scread+0x2d
giant_read(c5380500,ebdd4c60,0,0,1,...) at giant_read+0x48
devfs_read_f(c5873c18,ebdd4c60,c511c800,0,c5a39440,...) at devfs_read_f+0x71
dofileread(ebdd4c60,ffffffff,ffffffff,0,c5873c18,...) at dofileread+0x96
kern_readv(c5a39440,0,ebdd4c60,bfbfee4b,1,...) at kern_readv+0x58
read(c5a39440,ebdd4cfc,c,c0ace8dc,c0b49248,...) at read+0x4f
syscall(ebdd4d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x281644a3, esp =
0xbfbfee2c, ebp = 0xbfbfee58 ---
Tracing command getty pid 788 tid 100075 td 0xc55d5880
sched_switch(c55d5880,0,1,180,39b9a5dc,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c55d5880,0,c0aa02cc,19b,c538a810,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba580,1,c0a9b5fa,59,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c538a810,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(c538a810,0,159,c0aa3c6f,0,...) at _sleep+0x359
ttysleep(c538a800,c538a810,159,c0aa3c6f,0,...) at ttysleep+0x39
ttread(c538a800,ebcb7c60,0,ebcb7b9c,c0679bfd,...) at ttread+0x4e7
ttyread(c5380400,ebcb7c60,0,ebcb7bb8,c071e7e8,...) at ttyread+0x39
scread(c5380400,ebcb7c60,0,168,0,...) at scread+0x2d
giant_read(c5380400,ebcb7c60,0,0,1,...) at giant_read+0x48
devfs_read_f(c55a6510,ebcb7c60,c511c800,0,c55d5880,...) at devfs_read_f+0x71
dofileread(ebcb7c60,ffffffff,ffffffff,0,c55a6510,...) at dofileread+0x96
kern_readv(c55d5880,0,ebcb7c60,bfbfee4b,1,...) at kern_readv+0x58
read(c55d5880,ebcb7cfc,c,c0ace8dc,c0b49248,...) at read+0x4f
syscall(ebcb7d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x281644a3, esp =
0xbfbfee2c, ebp = 0xbfbfee58 ---
Tracing command getty pid 787 tid 100087 td 0xc562c440
sched_switch(c562c440,0,1,180,396e591c,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c562c440,0,c0aa02cc,19b,c5388c10,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba600,1,c0a9b5fa,59,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c5388c10,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(c5388c10,0,159,c0aa3c6f,0,...) at _sleep+0x359
ttysleep(c5388c00,c5388c10,159,c0aa3c6f,0,...) at ttysleep+0x39
ttread(c5388c00,ebd01c60,0,ebd01b9c,c0679bfd,...) at ttread+0x4e7
ttyread(c5380300,ebd01c60,0,ebd01bb8,c071e7e8,...) at ttyread+0x39
scread(c5380300,ebd01c60,0,168,0,...) at scread+0x2d
giant_read(c5380300,ebd01c60,0,0,1,...) at giant_read+0x48
devfs_read_f(c5873cf0,ebd01c60,c511c800,0,c562c440,...) at devfs_read_f+0x71
dofileread(ebd01c60,ffffffff,ffffffff,0,c5873cf0,...) at dofileread+0x96
kern_readv(c562c440,0,ebd01c60,bfbfee4b,1,...) at kern_readv+0x58
read(c562c440,ebd01cfc,c,c0ace8dc,c0b49248,...) at read+0x4f
syscall(ebd01d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x281644a3, esp =
0xbfbfee2c, ebp = 0xbfbfee58 ---
Tracing command getty pid 786 tid 100091 td 0xc562baa0
sched_switch(c562baa0,0,1,180,394c38d0,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c562baa0,0,c0aa02cc,19b,c5388810,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba600,1,c0a9b5fa,59,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c5388810,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(c5388810,0,159,c0aa3c6f,0,...) at _sleep+0x359
ttysleep(c5388800,c5388810,159,c0aa3c6f,0,...) at ttysleep+0x39
ttread(c5388800,ebcf5c60,0,ebcf5b9c,c0679bfd,...) at ttread+0x4e7
ttyread(c5380200,ebcf5c60,0,ebcf5bb8,c071e7e8,...) at ttyread+0x39
scread(c5380200,ebcf5c60,0,168,0,...) at scread+0x2d
giant_read(c5380200,ebcf5c60,0,0,1,...) at giant_read+0x48
devfs_read_f(c55cb630,ebcf5c60,c511c800,0,c562baa0,...) at devfs_read_f+0x71
dofileread(ebcf5c60,ffffffff,ffffffff,0,c55cb630,...) at dofileread+0x96
kern_readv(c562baa0,0,ebcf5c60,bfbfee4b,1,...) at kern_readv+0x58
read(c562baa0,ebcf5cfc,c,c0ace8dc,c0b49248,...) at read+0x4f
syscall(ebcf5d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x281644a3, esp =
0xbfbfee2c, ebp = 0xbfbfee58 ---
Tracing command getty pid 785 tid 100085 td 0xc52d5440
sched_switch(c52d5440,0,1,180,f4cd50d8,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c52d5440,0,c0aa02cc,19b,c5389810,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba580,1,c0a9b5fa,59,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c5389810,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(c5389810,0,159,c0aa3c6f,0,...) at _sleep+0x359
ttysleep(c5389800,c5389810,159,c0aa3c6f,0,...) at ttysleep+0x39
ttread(c5389800,e99b4c60,0,e99b4b9c,c0679bfd,...) at ttread+0x4e7
ttyread(c5380100,e99b4c60,0,e99b4bb8,c071e7e8,...) at ttyread+0x39
scread(c5380100,e99b4c60,0,168,0,...) at scread+0x2d
giant_read(c5380100,e99b4c60,0,0,1,...) at giant_read+0x48
devfs_read_f(c5634510,e99b4c60,c511c800,0,c52d5440,...) at devfs_read_f+0x71
dofileread(e99b4c60,ffffffff,ffffffff,0,c5634510,...) at dofileread+0x96
kern_readv(c52d5440,0,e99b4c60,bfbfee4b,1,...) at kern_readv+0x58
read(c52d5440,e99b4cfc,c,c0ace8dc,c0b49248,...) at read+0x4f
syscall(e99b4d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x281644a3, esp =
0xbfbfee2c, ebp = 0xbfbfee58 ---
Tracing command inetd pid 764 tid 100094 td 0xc562b440
sched_switch(c562b440,0,1,180,1f622752,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c562b440,0,c0aa02cc,19b,c0c03cbc,...) at sleepq_switch+0xf0
sleepq_catch_signals(c0742baa,c0c03ca4,0,c0a9b5fa,c562b440,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c0c03cbc,c0c03ca4,c0ab9fa0,101,0,...) at
sleepq_wait_sig+0x14
_cv_wait_sig(c0c03cbc,c0c03ca4,c0aa1e52,30f,ebcecb18,...) at
_cv_wait_sig+0x1fc
kern_select(c562b440,6,bfbfec8c,0,0,0,c58af288,bfbfee48) at
kern_select+0x6cc
select(c562b440,ebceccfc,14,c0aa2454,c0b49ab8,...) at select+0x5e
syscall(ebcecd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x28177423, esp =
0xbfbfe08c, ebp = 0xbfbfee78 ---
Tracing command moused pid 737 tid 100089 td 0xc562c000
sched_switch(c562c000,0,1,180,b377436,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c562c000,0,c0aa02cc,19b,c0c03cbc,...) at sleepq_switch+0xf0
sleepq_catch_signals(c0742baa,c0c03ca4,0,c0a9b5fa,c562c000,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c0c03cbc,c0c03ca4,c0ab9fa0,101,0,...) at
sleepq_wait_sig+0x14
_cv_wait_sig(c0c03cbc,c0c03ca4,c0aa1e52,30f,c065563f,...) at
_cv_wait_sig+0x1fc
kern_select(c562c000,400,bfbfeaac,0,0,0,4739b5e9,0) at kern_select+0x6cc
select(c562c000,ebcfbcfc,14,c0ace8dc,c0b49ab8,...) at select+0x5e
syscall(ebcfbd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x2817c423, esp =
0xbfbfea2c, ebp = 0xbfbfebc8 ---
Tracing command cron pid 721 tid 100086 td 0xc52d5220
sched_switch(c52d5220,0,1,180,49e8e632,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c52d5220,0,c0aa02cc,19b,c0bb2724,...) at sleepq_switch+0xf0
sleepq_catch_signals(e99b1bc4,c074280d,c0bba580,1,5c,...) at
sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0bb2724,0,c0a9d1f9,da,0,...) at
sleepq_timedwait_sig+0x17
_sleep(c0bb2724,0,15c,c0a9d9d5,ea61,...) at _sleep+0x331
kern_nanosleep(c52d5220,e99b1c64,e99b1c6c,3c,0,...) at kern_nanosleep+0xc1
nanosleep(c52d5220,e99b1cfc,8,c0aa2545,c0b4a880,...) at nanosleep+0x6f
syscall(e99b1d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x28150f7f, esp =
0xbfbfed0c, ebp = 0xbfbfed38 ---
Tracing command sendmail pid 715 tid 100047 td 0xc5255aa0
sched_switch(c5255aa0,0,1,180,3d3fe238,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5255aa0,0,c0aa02cc,19b,c52d7da8,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba600,1,c0a9b5fa,68,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c52d7da8,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(c52d7da8,c52d7dd8,168,c0a5f873,0,...) at _sleep+0x359
kern_sigsuspend(c5255aa0,0,0,0,0,...) at kern_sigsuspend+0xe4
sigsuspend(c5255aa0,e5757cfc,4,c0aa22db,c0b4b1f8,...) at sigsuspend+0x4d
syscall(e5757d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (341, FreeBSD ELF32, sigsuspend), eip = 0x2830bffb, esp =
0xbfbfdb1c, ebp = 0xbfbfdb48 ---
Tracing command sendmail pid 711 tid 100071 td 0xc559e440
sched_switch(c559e440,0,1,180,7f893e1c,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c559e440,0,c0aa02cc,19b,c0c03cbc,...) at sleepq_switch+0xf0
sleepq_catch_signals(ebc61a84,c0742baa,c0c03ca4,0,c559e440,...) at
sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0c03cbc,1389,c0ab9fa0,101,0,...) at
sleepq_timedwait_sig+0x17
_cv_timedwait_sig(c0c03cbc,c0c03ca4,1389,30f,c073fd6d,...) at
_cv_timedwait_sig+0x20b
kern_select(c559e440,5,bfbfd92c,0,0,ebc61c70,5,0) at kern_select+0x6b4
select(c559e440,ebc61cfc,14,c0aa27fa,c0b49ab8,...) at select+0x5e
syscall(ebc61d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x2839d423, esp =
0xbfbfd09c, ebp = 0xbfbfdb38 ---
Tracing command sshd pid 705 tid 100081 td 0xc55d2aa0
sched_switch(c55d2aa0,0,1,180,e160d12c,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c55d2aa0,0,c0aa02cc,19b,c0c03cbc,...) at sleepq_switch+0xf0
sleepq_catch_signals(c0742baa,c0c03ca4,0,c0a9b5fa,c55d2aa0,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c0c03cbc,c0c03ca4,c0ab9fa0,101,0,...) at
sleepq_wait_sig+0x14
_cv_wait_sig(c0c03cbc,c0c03ca4,c0aa1e52,30f,ebca5afc,...) at
_cv_wait_sig+0x1fc
kern_select(c55d2aa0,7,2850b0b8,0,0,0,c,ebca5c98) at kern_select+0x6cc
select(c55d2aa0,ebca5cfc,14,ebca5d38,c0b49ab8,...) at select+0x5e
syscall(ebca5d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x283e3423, esp =
0xh(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5255cc0,0,c0aa02cc,19b,c0c03cbc,...) at sleepq_switch+0xf0
sleepq_catch_signals(c0742baa,c0c03ca4,0,c0a9b5fa,c5255cc0,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c0c03cbc,c0c03ca4,c0ab9fa0,101,0,...) at
sleepq_wait_sig+0x14
_cv_wait_sig(c0c03cbc,c0c03ca4,c0aa1e52,30f,0,...) at _cv_wait_sig+0x1fc
kern_select(c5255cc0,9,282200b8,0,0,0,396c5,28177818) at kern_select+0x6cc
select(c5255cc0,e575acfc,14,c0aa23f0,c0b49ab8,...) at select+0x5e
syscall(e575ad38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x28167423, esp =
0xbfbfdf3c, ebp = 0xbfbfee88 ---
Tracing command devd pid 524 tid 100048 td 0xc5255880
sched_switch(c5255880,0,1,180,88841474,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5255880,0,c0aa02cc,19b,c0c03cbc,...) at sleepq_switch+0xf0
sleepq_catch_signals(c0742baa,c0c03ca4,0,c0a9b5fa,c5255880,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c0c03cbc,c0c03ca4,c0ab9fa0,101,0,...) at
sleepq_wait_sig+0x14
_cv_wait_sig(c0c03cbc,c0c03ca4,c0aa1e52,30f,e5754afc,...) at
_cv_wait_sig+0x1fc
kern_select(c5255880,5,bfbfedf8,0,0,0,0,bfbfe99c) at kern_select+0x6cc
select(c5255880,e5754cfc,14,c0aa2748,c0b49ab8,...) at select+0x5e
syscall(e5754d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x807f95f, esp =
0xbfbfe9bc, ebp = 0xbfbfee98 ---
Tracing command dhclient pid 328 tid 100084 td 0xc52d5660
sched_switch(c52d5660,0,1,180,3bd05a58,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c52d5660,0,c0aa02cc,19b,c0c03cbc,...) at sleepq_switch+0xf0
sleepq_catch_signals(e99b7ad8,c0742baa,c0c03ca4,0,c52d5660,...) at
sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0c03cbc,14ff1,c0ab9fa0,101,0,...) at
sleepq_timedwait_sig+0x17
_cv_timedwait_sig(c0c03cbc,c0c03ca4,14ff1,3bb,e99b7b48,...) at
_cv_timedwait_sig+0x20b
poll(c52d5660,e99b7cfc,c,c0a5612b,c0b4a598,...) at poll+0x554
syscall(e99b7d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (209, FreeBSD ELF32, poll), eip = 0x2810f9af, esp =
0xbfbfee1c, ebp = 0xbfbfee48 ---
Tracing command dhclient pid 308 tid 100095 td 0xc562b220
sched_switch(c562b220,0,1,180,50643cea,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c562b220,0,c0aa02cc,19b,c0c03cbc,...) at sleepq_switch+0xf0
sleepq_catch_signals(c0742baa,c0c03ca4,0,c0a9b5fa,c562b220,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c0c03cbc,c0c03ca4,c0ab9fa0,101,0,...) at
sleepq_wait_sig+0x14
_cv_wait_sig(c0c03cbc,c0c03ca4,c0aa1e52,3bb,c0aa1eef,...) at
_cv_wait_sig+0x1fc
poll(c562b220,ebce9cfc,c,c0a8d525,c0b4a598,...) at poll+0x56a
syscall(ebce9d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (209, FreeBSD ELF32, poll), eip = 0x2810f9af, esp =
0xbfbfee1c, ebp = 0xbfbfee48 ---
Tracing command softdepflush pid 53 tid 100049 td 0xc5255660
sched_switch(c5255660,0,1,180,9dbda24,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,44,...) at mi_switch+0x217
sleepq_switch(c5255660,0,c0aa02cc,240,44,...) at sleepq_switch+0xf0
sleepq_timedwait(c0c0f9a4,3e8,c0abbaca,0,0,...) at sleepq_timedwait+0x68
_sleep(c0c0f9a4,c0c0f97c,44,c0abbaca,3e8,...) at _sleep+0x345
softdep_flush(0,e5751d38,c0a9920b,2ea,c54f92a8,...) at softdep_flush+0x2c8
fork_exit(c09354b0,0,e5751d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe5751d70, ebp = 0 ---
Tracing command syncer pid 52 tid 100050 td 0xc54facc0
sched_switch(c54facc0,0,1,180,a09f520,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,c54facc0,...) at mi_switch+0x217
sleepq_switch(c54facc0,0,c0aa02cc,21d,68,...) at sleepq_switch+0xf0
sleepq_wait(c0bb254c,0,c0aa8698,0,0,...) at sleepq_wait+0x60
_sleep(c0bb254c,c0c04444,68,c0aa8698,0,...) at _sleep+0x369
sched_sync(0,e9a69d38,c0a9920b,2ea,c54f9550,...) at sched_sync+0x56e
fork_exit(c07cb690,0,e9a69d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe9a69d70, ebp = 0 ---
Tracing command vnlru pid 51 tid 100051 td 0xc54faaa0
sched_switch(c54faaa0,0,1,180,49fd376c,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,50,...) at mi_switch+0x217
sleepq_switch(c54faaa0,0,c0aa02cc,240,50,...) at sleepq_switch+0xf0
sleepq_timedwait(c54f97f8,3e8,c0aa8742,0,0,...) at sleepq_timedwait+0x68
_sleep(c54f97f8,c0c04418,250,c0aa8742,3e8,...) at _sleep+0x345
vnlru_proc(0,e9a66d38,c0a9920b,2ea,c54f97f8,...) at vnlru_proc+0x114
fork_exit(c07cc450,0,e9a66d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe9a66d70, ebp = 0 ---
Tracing command bufdaemon pid 50 tid 100052 td 0xc54fa880
sched_switch(c54fa880,0,1,180,da7be8b8,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,44,...) at mi_switch+0x217
sleepq_switch(c54fa880,0,c0aa02cc,240,44,...) at sleepq_switch+0xf0
sleepq_timedwait(c0c04164,3e8,c0aa6585,0,0,...) at sleepq_timedwait+0x68
_sleep(c0c04164,c0c04168,44,c0aa6585,3e8,...) at _sleep+0x345
buf_daemon(0,e9a63d38,c0a9920b,2ea,c54f9aa0,...) at buf_daemon+0x21e
fork_exit(c07b8280,0,e9a63d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe9a63d70, ebp = 0 ---
Tracing command pagezero pid 49 tid 100053 td 0xc54fa660
sched_switch(c54fa660,0,1,180,6413cea6,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c54fa660,0,c0aa02cc,240,0,...) at sleepq_switch+0xf0
sleepq_timedwait(c0c10560,493e0,c0ac062b,0,0,...) at sleepq_timedwait+0x68
_sleep(c0c10560,c0c10104,0,c0ac062b,493e0,...) at _sleep+0x345
vm_pagezero(0,e9a60d38,c0a9920b,2ea,c54f9d48,...) at vm_pagezero+0xdc
fork_exit(c096ea00,0,e9a60d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe9a60d70, ebp = 0 ---
Tracing command vmdaemon pid 48 tid 100054 td 0xc54fa440
sched_switch(c54fa440,0,1,180,4968ab00,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,c54fa440,...) at mi_switch+0x217
sleepq_switch(c54fa440,0,c0aa02cc,21d,68,...) at sleepq_switch+0xf0
sleepq_wait(c0c10178,0,c0a9d1f9,da,0,...) at sleepq_wait+0x60
_sleep(c0c10178,c0c1017c,68,c0aa6585,0,...) at _sleep+0x369
vm_daemon(0,e9a5dd38,c0a9920b,2ea,c54fb000,...) at vm_daemon+0x59
fork_exit(c0969e40,0,e9a5dd38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe9a5dd70, ebp = 0 ---
Tracing command pagedaemon pid 47 tid 100055 td 0xc54fa220
sched_switch(c54fa220,0,1,180,502fd750,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,44,...) at mi_switch+0x217
sleepq_switch(c54fa220,0,c0aa02cc,240,44,...) at sleepq_switch+0xf0
sleepq_timedwait(c0c10140,1388,c0aa6585,0,0,...) at sleepq_timedwait+0x68
_sleep(c0c10140,c0c10104,44,c0aa6585,1388,...) at _sleep+0x345
vm_pageout(0,e9a5ad38,c0a9920b,2ea,c54fb2a8,...) at vm_pageout+0x2bb
fork_exit(c096a9f0,0,e9a5ad38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe9a5ad70, ebp = 0 ---
Tracing command sctp_iterator pid 46 tid 100056 td 0xc54fa000
sched_switch(c54fa000,0,1,180,16b33bc0,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,c54fa000,...) at mi_switch+0x217
sleepq_switch(c54fa000,0,c0aa02cc,21d,0,...) at sleepq_switch+0xf0
sleepq_wait(c0c05ecc,c0c05dd4,c0aaf4d9,0,0,...) at sleepq_wait+0x60
_sleep(c0c05ecc,c0c05dd4,0,c0aaf4d9,0,...) at _sleep+0x369
sctp_iterator_thread(0,e9a6cd38,c0a9920b,2ea,c54fb550,...) at
sctp_iterator_thread+0x60
fork_exit(c0833730,0,e9a6cd38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe9a6cd70, ebp = 0 ---
Tracing command irq7: ppc0 pid 45 tid 100034 td 0xc5254440
fork_trampoline() at fork_trampoline
Tracing command fdc0 pid 44 tid 100035 td 0xc5254220
sched_switch(c5254220,0,1,180,9d4623c,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,4c,...) at mi_switch+0x217
sleepq_switch(c5254220,0,c0aa02cc,240,4c,...) at sleepq_switch+0xf0
sleepq_timedwait(c5324e3c,3e8,c0a94196,0,0,...) at sleepq_timedwait+0x68
_sleep(c5324e3c,c5324ef0,4c,c0a94196,3e8,...) at _sleep+0x345
fdc_thread(c5324e00,e5736d38,c0a9920b,2ea,c5256d48,...) at fdc_thread+0x2b8
fork_exit(c09c69b0,c5324e00,e5736d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe5736d70, ebp = 0 ---
Tracing command swi0: sio pid 43 tid 100036 td 0xc5254000
sched_switch(c5254000,0,1,180,37bd6d44,...) at sched_switch+0x49b
mi_switch(1,0,c0a99497,46b,c53629e4,...) at mi_switch+0x217
ithread_loop(c5360b50,e5733d38,c0a9920b,2ea,c52d3000,...) at
ithread_loop+0x34c
fork_exit(c0732ea0,c5360b50,e5733d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe5733d70, ebp = 0 ---
Tracing command irq12: psm0 pid 42 tid 100037 td 0xc5160cc0
fork_trampoline() at fork_trampoline
Tracing command irq1: atkbd0 pid 41 tid 100038 td 0xc52d5000
sched_switch(c52d5000,0,1,180,4f6f164,...) at sched_switch+0x49b
mi_switch(1,0,c0a99497,46b,c5111464,...) at mi_switch+0x217
ithread_loop(c5379340,e99aed38,c0a9920b,2ea,c52d3550,...) at
ithread_loop+0x34c
fork_exit(c0732ea0,c5379340,e99aed38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe99aed70, ebp = 0 ---
Tracing command irq15: ata1 pid 40 tid 100039 td 0xc52d4cc0
sched_switch(c52d4cc0,0,1,180,4a2ac8a4,...) at sched_switch+0x49b
mi_switch(1,0,c0a99497,46b,c5110c64,...) at mi_switch+0x217
ithread_loop(c535bab0,e99abd38,c0a9920b,2ea,c52d37f8,...) at
ithread_loop+0x34c
fork_exit(c0732ea0,c535bab0,e99abd38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe99abd70, ebp = 0 ---
Tracing command irq14: ata0 pid 39 tid 100040 td 0xc52d4aa0
fork_trampoline() at fork_trampoline
Tracing command aic_recovery1 pid 38 tid 100041 td 0xc52d4880
sched_switch(c52d4880,0,1,180,8167da80,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,c52d4880,...) at mi_switch+0x217
sleepq_switch(c52d4880,0,c0aa02cc,21d,a0,...) at sleepq_switch+0xf0
sleepq_wait(c5326000,0,c0a9d1f9,da,0,...) at sleepq_wait+0x60
_sleep(c5326000,c53212b4,a0,c0a5f5b8,0,...) at _sleep+0x369
ahd_recovery_thread(c5326000,e99a5d38,c0a9920b,2ea,c52d3d48,...) at
ahd_recovery_thread+0x72
fork_exit(c04c3d60,c5326000,e99a5d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe99a5d70, ebp = 0 ---
Tracing command irq22: ahd1 pid 37 tid 100042 td 0xc52d4660
sched_switch(c52d4660,0,1,180,45c9b3ec,...) at sched_switch+0x49b
mi_switch(1,0,c0a99497,46b,c515e4e4,...) at mi_switch+0x217
ithread_loop(c535b470,e99a2d38,c0a9920b,2ea,c52d7000,...) at
ithread_loop+0x34c
fork_exit(c0732ea0,c535b470,e99a2d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe99a2d70, ebp = 0 ---
Tracing command aic_recovery0 pid 36 tid 100043 td 0xc52d4440
sched_switch(c52d4440,0,1,180,8167a830,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,c52d4440,...) at mi_switch+0x217
sleepq_switch(c52d4440,0,c0aa02cc,21d,a0,...) at sleepq_switch+0xf0
sleepq_wait(c52db000,0,c0a9d1f9,da,0,...) at sleepq_wait+0x60
_sleep(c52db000,c5265bb4,a0,c0a5f5b8,0,...) at _sleep+0x369
ahd_recovery_thread(c52db000,e999fd38,c0a9920b,2ea,c52d72a8,...) at
ahd_recovery_thread+0x72
fork_exit(c04c3d60,c52db000,e999fd38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe999fd70, ebp = 0 ---
Tracing command aac0aif pid 35 tid 100044 td 0xc52d4220
sched_switch(c52d4220,0,1,180,deb41c52,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,4c,...) at mi_switch+0x217
sleepq_switch(c52d4220,0,c0aa02cc,240,4c,...) at sleepq_switch+0xf0
sleepq_timedwait(c52d7550,4e20,c0a5acdf,0,0,...) at sleepq_timedwait+0x68
_sleep(c52d7550,c5276128,4c,c0a5acdf,4e20,...) at _sleep+0x345
aac_command_thread(c5276000,e999cd38,c0a9920b,2ea,c52d7550,...) at
aac_command_thread+0x8d
fork_exit(c0496140,c5276000,e999cd38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe999cd70, ebp = 0 ---
Tracing command irq18: aac0 pid 34 tid 100024 td 0xc5119aa0
sched_switch(c5119aa0,0,1,180,fc64d9ac,...) at sched_switch+0x49b
mi_switch(1,0,c0a99497,46b,c515e6e4,...) at mi_switch+0x217
ithread_loop(c5271a50,e3cd2d38,c0a9920b,2ea,c5251000,...) at
ithread_loop+0x34c
fork_exit(c0732ea0,c5271a50,e3cd2d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe3cd2d70, ebp = 0 ---
Tracing command irq21: bge0 pid 33 tid 100025 td 0xc5119880
cpustop_handler(1,e3ccf958,c0a05b10,1,0,...) at cpustop_handler+0x32
ipi_nmi_handler(1,0,0,0,13,...) at ipi_nmi_handler+0x2f
trap(e3ccf964) at trap+0x30
calltrap() at calltrap+0x6
--- trap 0x13, eip = 0xc074e586, esp = 0xe3ccf9a4, ebp = 0xe3ccf9c8 ---
panic(c0a9b762,c5117660,c0aa0a06,c5117660,186ae,...) at panic+0x26
_mtx_lock_spin_failed(1,19,c0aa0a32,cb,19,...) at _mtx_lock_spin_failed+0x51
_thread_lock_flags(c57c6440,10,c0aa0a32,cb,1,...) at _thread_lock_flags+0xc7
propagate_priority(c0bbceec,0,c0aa0a32,2e2,c50f2a00,...) at
propagate_priority+0xe0
turnstile_wait(c50f2a00,c57c6440,0,17a,c5874678,...) at turnstile_wait+0x48c
_mtx_lock_sleep(c5874678,c5119880,0,c0aaa355,41c,...) at
_mtx_lock_sleep+0x15a
_mtx_lock_flags(c5874678,0,c0aaa355,41c,e3ccfaf4,...) at
_mtx_lock_flags+0xef
rt_setgate(c5874618,cb4ee2c0,e3ccfbac,c5119880,e3ccfb18,...) at
rt_setgate+0x196
rtredirect(e3ccfbbc,e3ccfbac,0,26,e3ccfb9c,...) at rtredirect+0x18e
icmp_input(c6fb6500,14,246,c0b9c7c0,e3ccfc0c,...) at icmp_input+0x50f
ip_input(c6fb6500,14e,800,c526a400,800,...) at ip_input+0x650
netisr_dispatch(2,c6fb6500,10,3,0,...) at netisr_dispatch+0x73
ether_demux(c526a400,c6fb6500,3,0,3,...) at ether_demux+0x1f1
ether_input(c526a400,c6fb6500,c0a6af05,bc4,c526a400,...) at
ether_input+0x37f
bge_intr(c526d000,0,c0a99497,471,c515e564,...) at bge_intr+0x7da
ithread_loop(c5271160,e3ccfd38,c0a9920b,2ea,c52512a8,...) at
ithread_loop+0x1b5
fork_exit(c0732ea0,c5271160,e3ccfd38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe3ccfd70, ebp = 0 ---
Tracing command usb2 pid 32 tid 100026 td 0xc5119660
sched_switch(c5119660,0,1,180,5e0d4486,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,5c,...) at mi_switch+0x217
sleepq_switch(c5119660,0,c0aa02cc,240,5c,...) at sleepq_switch+0xf0
sleepq_timedwait(c520ba10,ea60,c0a9111c,0,0,...) at sleepq_timedwait+0x68
_sleep(c520ba10,0,5c,c0a9111c,ea60,...) at _sleep+0x345
usb_event_thread(c52685c0,e3cccd38,c0a9920b,2ea,c5251550,...) at
usb_event_thread+0xca
fork_exit(c06cbb40,c52685c0,e3cccd38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe3cccd70, ebp = 0 ---
Tracing command irq23: ehci0 pid 31 tid 100027 td 0xc5255440
fork_trampoline() at fork_trampoline
Tracing command usb1 pid 30 tid 100028 td 0xc5255220
sched_switch(c5255220,0,1,180,5e0e59ae,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,5c,...) at mi_switch+0x217
sleepq_switch(c5255220,0,c0aa02cc,240,5c,...) at sleepq_switch+0xf0
sleepq_timedwait(c5243210,ea60,c0a9111c,0,0,...) at sleepq_timedwait+0x68
_sleep(c5243210,0,5c,c0a9111c,ea60,...) at _sleep+0x345
usb_event_thread(c523cc00,e574bd38,c0a9920b,2ea,c5251aa0,...) at
usb_event_thread+0xca
fork_exit(c06cbb40,c523cc00,e574bd38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe574bd70, ebp = 0 ---
Tracing command irq19: uhci1 ahd0 pid 29 tid 100029 td 0xc5255000
sched_switch(c5255000,0,1,180,2f1617b8,...) at sched_switch+0x49b
mi_switch(1,0,c0a99497,46b,c515e664,...) at mi_switch+0x217
ithread_loop(c5242a20,e5748d38,c0a9920b,2ea,c5251d48,...) at
ithread_loop+0x34c
fork_exit(c0732ea0,c5242a20,e5748d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe5748d70, ebp = 0 ---
Tracing command usbtask-dr pid 28 tid 100030 td 0xc5254cc0
sched_switch(c5254cc0,0,1,180,81669054,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,c5254cc0,...) at mi_switch+0x217
sleepq_switch(c5254cc0,0,c0aa02cc,21d,5c,...) at sleepq_switch+0xf0
sleepq_wait(c0bafe54,0,c0a9d1f9,da,0,...) at sleepq_wait+0x60
_sleep(c0bafe54,0,5c,c0a91159,0,...) at _sleep+0x369
usb_task_thread(c0bafe54,e5745d38,c0a9920b,2ea,c5256000,...) at
usb_task_thread+0x62
fork_exit(c06cc2c0,c0bafe54,e5745d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe5745d70, ebp = 0 ---
Tracing command usbtask-hc pid 27 tid 100031 td 0xc5254aa0
sched_switch(c5254aa0,0,1,180,816663d8,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,c5254aa0,...) at mi_switch+0x217
sleepq_switch(c5254aa0,0,c0aa02cc,21d,5c,...) at sleepq_switch+0xf0
sleepq_wait(c0bafe40,0,c0a9d1f9,da,0,...) at sleepq_wait+0x60
_sleep(c0bafe40,0,5c,c0a91159,0,...) at _sleep+0x369
usb_task_thread(c0bafe40,e5742d38,c0a9920b,2ea,c52562a8,...) at
usb_task_thread+0x62
fork_exit(c06cc2c0,c0bafe40,e5742d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe5742d70, ebp = 0 ---
Tracing command usb0 pid 26 tid 100032 td 0xc5254880
sched_switch(c5254880,0,1,180,5e0ecc9a,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,5c,...) at mi_switch+0x217
sleepq_switch(c5254880,0,c0aa02cc,240,5c,...) at sleepq_switch+0xf0
sleepq_timedwait(c5249210,ea60,c0a9111c,0,0,...) at sleepq_timedwait+0x68
_sleep(c5249210,0,5c,c0a9111c,ea60,...) at _sleep+0x345
usb_event_thread(c524d680,e573fd38,c0a9920b,2ea,c5256550,...) at
usb_event_thread+0xca
fork_exit(c06cbb40,c524d680,e573fd38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe573fd70, ebp = 0 ---
Tracing command irq16: uhci0 pid 25 tid 100033 td 0xc5254660
fork_trampoline() at fork_trampoline
Tracing command irq9: acpi0 pid 24 tid 100015 td 0xc5117440
fork_trampoline() at fork_trampoline
Tracing command kqueue taskq pid 23 tid 100016 td 0xc5117220
sched_switch(c5117220,0,1,180,81b1c448,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,c5117220,...) at mi_switch+0x217
sleepq_switch(c5117220,0,c0aa02cc,21d,0,...) at sleepq_switch+0xf0
sleepq_wait(c5213600,c521361c,c0a94196,0,0,...) at sleepq_wait+0x60
_sleep(c5213600,c521361c,0,c0a94196,0,...) at _sleep+0x369
taskqueue_thread_loop(c0bb0dc4,e3cb1d38,c0a9920b,2ea,c515faa0,...) at
taskqueue_thread_loop+0xb4
fork_exit(c0780830,c0bb0dc4,e3cb1d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe3cb1d70, ebp = 0 ---
Tracing command swi6: task queue pid 22 tid 100017 td 0xc5160aa0
sched_switch(c5160aa0,0,1,180,1fb23b70,...) at sched_switch+0x49b
mi_switch(1,0,c0a99497,46b,c52136e4,...) at mi_switch+0x217
ithread_loop(c520a0e0,e570ad38,c0a9920b,2ea,c515fd48,...) at
ithread_loop+0x34c
fork_exit(c0732ea0,c520a0e0,e570ad38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe570ad70, ebp = 0 ---
Tracing command swi6: Giant taskq pid 21 tid 100018 td 0xc5160880
sched_switch(c5160880,0,1,180,4f7572c,...) at sched_switch+0x49b
mi_switch(1,0,c0a99497,46b,c52137e4,...) at mi_switch+0x217
ithread_loop(c520a0f0,e5707d38,c0a9920b,2ea,c5214000,...) at
ithread_loop+0x34c
fork_exit(c0732ea0,c520a0f0,e5707d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe5707d70, ebp = 0 ---
Tracing command thread taskq pid 9 tid 100019 td 0xc5160660
sched_switch(c5160660,0,1,180,45ed1704,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,c5160660,...) at mi_switch+0x217
sleepq_switch(c5160660,0,c0aa02cc,21d,0,...) at sleepq_switch+0xf0
sleepq_wait(c5213880,c521389c,c0a94196,0,0,...) at sleepq_wait+0x60
_sleep(c5213880,c521389c,0,c0a94196,0,...) at _sleep+0x369
taskqueue_thread_loop(c0bbc6d8,e5704d38,c0a9920b,2ea,c52142a8,...) at
taskqueue_thread_loop+0xb4
fork_exit(c0780830,c0bbc6d8,e5704d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe5704d70, ebp = 0 ---
Tracing command acpi_task_2 pid 8 tid 100020 td 0xc5160440
sched_switch(c5160440,0,1,180,817489b8,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,c5160440,...) at mi_switch+0x217
sleepq_switch(c5160440,0,c0aa02cc,21d,0,...) at sleepq_switch+0xf0
sleepq_wait(c5213900,c521391c,c0a94196,0,0,...) at sleepq_wait+0x60
_sleep(c5213900,c521391c,0,c0a94196,0,...) at _sleep+0x369
taskqueue_thread_loop(c0d82ebc,e5701d38,c0a9920b,2ea,c5214550,...) at
taskqueue_thread_loop+0xb4
fork_exit(c0780830,c0d82ebc,e5701d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe5701d70, ebp = 0 ---
Tracing command acpi_task_1 pid 7 tid 100021 td 0xc5160220
sched_switch(c5160220,0,1,180,81745958,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,c5160220,...) at mi_switch+0x217
sleepq_switch(c5160220,0,c0aa02cc,21d,0,...) at sleepq_switch+0xf0
sleepq_wait(c5213900,c521391c,c0a94196,0,0,...) at sleepq_wait+0x60
_sleep(c5213900,c521391c,0,c0a94196,0,...) at _sleep+0x369
taskqueue_thread_loop(c0d82ebc,e56fed38,c0a9920b,2ea,c52147f8,...) at
taskqueue_thread_loop+0xb4
fork_exit(c0780830,c0d82ebc,e56fed38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe56fed70, ebp = 0 ---
Tracing command acpi_task_0 pid 6 tid 100022 td 0xc5160000
sched_switch(c5160000,0,1,180,817425a0,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,c5160000,...) at mi_switch+0x217
sleepq_switch(c5160000,0,c0aa02cc,21d,0,...) at sleepq_switch+0xf0
sleepq_wait(c5213900,c521391c,c0a94196,0,0,...) at sleepq_wait+0x60
_sleep(c5213900,c521391c,0,c0a94196,0,...) at _sleep+0x369
taskqueue_thread_loop(c0d82ebc,e56fbd38,c0a9920b,2ea,c5214aa0,...) at
taskqueue_thread_loop+0xb4
fork_exit(c0780830,c0d82ebc,e56fbd38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe56fbd70, ebp = 0 ---
Tracing command swi5: + pid 20 tid 100023 td 0xc5119cc0
fork_trampoline() at fork_trampoline
Tracing command swi2: cambio pid 19 tid 100007 td 0xc5115000
sched_switch(c5115000,0,1,180,45cc7cd4,...) at sched_switch+0x49b
mi_switch(1,0,c0a99497,46b,c5213ae4,...) at mi_switch+0x217
ithread_loop(c520a130,e3c99d38,c0a9920b,2ea,c51182a8,...) at
ithread_loop+0x34c
fork_exit(c0732ea0,c520a130,e3c99d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe3c99d70, ebp = 0 ---
Tracing command xpt_thrd pid 5 tid 100008 td 0xc5119440
sched_switch(c5119440,0,1,180,8165f1e0,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,c5119440,...) at mi_switch+0x217
sleepq_switch(c5119440,0,c0aa02cc,21d,4c,...) at sleepq_switch+0xf0
sleepq_wait(c0b81014,0,c0a9d1f9,da,0,...) at sleepq_wait+0x60
_sleep(c0b81014,c0b8102c,4c,c0a533ca,0,...) at _sleep+0x369
xpt_scanner_thread(0,e3cc9d38,c0a9920b,2ea,c5118550,...) at
xpt_scanner_thread+0x41
fork_exit(c04683a0,0,e3cc9d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe3cc9d70, ebp = 0 ---
Tracing command yarrow pid 18 tid 100009 td 0xc5119220
sched_switch(c5119220,0,1,180,9d6bbb0,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5119220,0,c0aa02cc,240,0,...) at sleepq_switch+0xf0
sleepq_timedwait(c0bb2554,64,c0a94196,2,0,...) at sleepq_timedwait+0x68
_sleep(c0bb2554,0,0,c0a94196,64,...) at _sleep+0x345
pause(c0a94196,64,c0a88aa0,112,0,...) at pause+0x47
random_kthread(0,e3cc6d38,c0a9920b,2ea,c51187f8,...) at random_kthread+0x1d9
fork_exit(c0656330,0,e3cc6d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe3cc6d70, ebp = 0 ---
Tracing command g_down pid 4 tid 100010 td 0xc5119000
sched_switch(c5119000,0,1,180,9d58524,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,4c,...) at mi_switch+0x217
sleepq_switch(c5119000,0,c0aa02cc,240,4c,...) at sleepq_switch+0xf0
sleepq_timedwait(c0bb05cc,64,c0a94196,0,0,...) at sleepq_timedwait+0x68
_sleep(c0bb05cc,c0bb04e8,24c,c0a94196,64,...) at _sleep+0x345
g_io_schedule_down(c5119000,0,c0a95891,74,0,...) at g_io_schedule_down+0x6b
g_down_procbody(0,e3cc3d38,c0a9920b,2ea,c5118aa0,...) at
g_down_procbody+0x8d
fork_exit(c0702520,0,e3cc3d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe3cc3d70, ebp = 0 ---
Tracing command g_up pid 3 tid 100011 td 0xc5117cc0
sched_switch(c5117cc0,0,1,180,da7c29c0,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,4c,...) at mi_switch+0x217
sleepq_switch(c5117cc0,0,c0aa02cc,240,4c,...) at sleepq_switch+0xf0
sleepq_timedwait(c0bb05c8,64,c0a94196,0,0,...) at sleepq_timedwait+0x68
_sleep(c0bb05c8,c0bb0528,24c,c0a94196,64,...) at _sleep+0x345
g_io_schedule_up(c5117cc0,0,c0a95891,5d,0,...) at g_io_schedule_up+0x133
g_up_procbody(0,e3cc0d38,c0a9920b,2ea,c5118d48,...) at g_up_procbody+0x8d
fork_exit(c07025b0,0,e3cc0d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe3cc0d70, ebp = 0 ---
Tracing command g_event pid 2 tid 100012 td 0xc5117aa0
sched_switch(c5117aa0,0,1,180,da7c8d90,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,4c,...) at mi_switch+0x217
sleepq_switch(c5117aa0,0,c0aa02cc,240,4c,...) at sleepq_switch+0xf0
sleepq_timedwait(c0bb05c0,64,c0a94196,0,0,...) at sleepq_timedwait+0x68
_sleep(c0bb05c0,0,4c,c0a94196,64,...) at _sleep+0x345
g_event_procbody(0,e3cbdd38,c0a9920b,2ea,c515f000,...) at
g_event_procbody+0xcb
fork_exit(c0702640,0,e3cbdd38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe3cbdd70, ebp = 0 ---
Tracing command swi3: vm pid 17 tid 100013 td 0xc5117880
fork_trampoline() at fork_trampoline
Tracing command swi4: clock sio pid 16 tid 100014 td 0xc5117660
kdb_enter(c0a9c88e,1,c0a9b762,e3cb7b7c,1,...) at kdb_enter+0x32
panic(c0a9b762,c5119880,c0aa0a06,c5119880,186b9,...) at panic+0x124
_mtx_lock_spin_failed(1,19,c0aa0a32,cb,19,...) at _mtx_lock_spin_failed+0x51
_thread_lock_flags(c5119880,10,c0aa0a32,cb,1,...) at _thread_lock_flags+0xc7
propagate_priority(c0bbc9c8,0,c0aa0a32,2e2,c51165f0,...) at
propagate_priority+0xe0
turnstile_wait(c51165f0,c57c6440,0,17a,c58a17c8,...) at turnstile_wait+0x48c
_mtx_lock_sleep(c58a17c8,c5117660,0,c0ab2bb5,1b6,...) at
_mtx_lock_sleep+0x15a
_mtx_lock_flags(c58a17c8,0,c0ab2bb5,1b6,412b9e7,...) at _mtx_lock_flags+0xef
tcp_timer_rexmt(c58a4378,0,c0a9daa7,ef,24,...) at tcp_timer_rexmt+0x96
softclock(0,0,c0a99497,471,c515e364,...) at softclock+0x266
ithread_loop(c5113280,e3cb7d38,c0a9920b,2ea,c515f550,...) at
ithread_loop+0x1b5
fork_exit(c0732ea0,c5113280,e3cb7d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe3cb7d70, ebp = 0 ---
Tracing command swi1: net pid 15 tid 100000 td 0xc5117000
sched_switch(c5117000,0,1,180,3bca4e84,...) at sched_switch+0x49b
mi_switch(1,0,c0a99497,46b,c515e3e4,...) at mi_switch+0x217
ithread_loop(c5113290,e3caed38,c0a9920b,2ea,c5114000,...) at
ithread_loop+0x34c
fork_exit(c0732ea0,c5113290,e3caed38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe3caed70, ebp = 0 ---
Tracing command idle: cpu0 pid 14 tid 100001 td 0xc5115cc0
sched_switch(c5115cc0,0,6,17b,3b3b9d38,...) at sched_switch+0x49b
mi_switch(6,0,c0ac91ab,495,0,...) at mi_switch+0x217
ipi_bitmap_handler(c5110008,e3ca0028,c5110028,c515c018,c515c000,...) at
ipi_bitmap_handler+0x72
Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0x2e
--- interrupt, eip = 0x20, esp = 0xc0d6b83c, ebp = 0xe3cabc7c ---
uart_z8530_class(c0d724a5,20,246,e3cabcc0,c0d6b83c,...) at 0x20
Tracing command idle: cpu1 pid 13 tid 100002 td 0xc5115aa0
sched_switch(c5115aa0,0,6,17b,9d3229c,...) at sched_switch+0x49b
mi_switch(6,0,c0ac91ab,495,1,...) at mi_switch+0x217
ipi_bitmap_handler(8,c0bb0028,28,0,f,...) at ipi_bitmap_handler+0x72
Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0x2e
--- interrupt, eip = 0x20, esp = 0xe3ca8cc8, ebp = 0xe3ca8ca8 ---
uart_z8530_class(c09f9bb2,20,202,2,e3ca8cc8,...) at 0x20
Tracing command idle: cpu2 pid 12 tid 100003 td 0xc5115880
cpustop_handler(4,e3ca5c3c,c0a05b10,c5115880,e3ca5bbc,...) at
cpustop_handler+0x32
ipi_nmi_handler(c5115880,e3ca5bbc,c076b378,e3ca5bf4,13,...) at
ipi_nmi_handler+0x2f
trap(e3ca5c48) at trap+0x30
calltrap() at calltrap+0x6
--- trap 0x13, eip = 0xc07431e5, esp = 0xe3ca5c88, ebp = 0xe3ca5ca8 ---
_mtx_lock_spin_flags(c0bba580,0,c0a9e25c,26d,c0bb47c0,...) at
_mtx_lock_spin_flags+0xd5
tdq_lock_pair(c0bba600,0,c0a9e25c,322,c0bb4e00,...) at tdq_lock_pair+0x82
sched_idletd(0,e3ca5d38,c0a9920b,2ea,c51147f8,...) at sched_idletd+0x19d
fork_exit(c076c300,0,e3ca5d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe3ca5d70, ebp = 0 ---
Tracing command idle: cpu3 pid 11 tid 100004 td 0xc5115660
cpustop_handler(8,e3ca2c70,c0a05b10,c076b378,e3ca2c20,...) at
cpustop_handler+0x32
ipi_nmi_handler(c076b378,e3ca2c20,c076bd94,c0bba600,13,...) at
ipi_nmi_handler+0x2f
trap(e3ca2c7c) at trap+0x30
calltrap() at calltrap+0x6
--- trap 0x13, eip = 0xc09f9bb2, esp = 0xe3ca2cbc, ebp = 0xe3ca2cc0 ---
mp_grab_cpu_hlt(e3ca2cf8,c076c549,c0bba600,0,c0a9e25c,...) at
mp_grab_cpu_hlt+0x22
cpu_idle(c0bba600,0,c0a9e25c,322,c0bb5440,...) at cpu_idle+0x8
sched_idletd(0,e3ca2d38,c0a9920b,2ea,c5114aa0,...) at sched_idletd+0x249
fork_exit(c076c300,0,e3ca2d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe3ca2d70, ebp = 0 ---
Tracing command init pid 1 tid 100005 td 0xc5115440
sched_switch(c5115440,0,1,180,e5706560,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5115440,0,c0aa02cc,19b,c5114d48,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074280d,c0bba600,1,c0a9b5fa,5c,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c5114d48,0,c0a9d1f9,da,0,...) at sleepq_wait_sig+0x14
_sleep(c5114d48,c5114dd8,15c,c0aa2401,0,...) at _sleep+0x359
kern_wait(c5115440,ffffffff,e3c9fc2c,0,0,...) at kern_wait+0xa66
wait4(c5115440,e3c9fcfc,10,c0aa2214,c0b492a8,...) at wait4+0x3b
syscall(e3c9fd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (7, FreeBSD ELF32, wait4), eip = 0x805452f, esp =
0xbfbfe97c, ebp = 0xbfbfe998 ---
Tracing command audit pid 10 tid 100006 td 0xc5115220
sched_switch(c5115220,0,1,180,815aaf9c,...) at sched_switch+0x49b
mi_switch(1,0,c0aa02cc,1bf,c5115220,...) at mi_switch+0x217
sleepq_switch(c5115220,0,c0aa02cc,21d,c5115220,...) at sleepq_switch+0xf0
sleepq_wait(c0c0f414,c0c0f3f4,c0ab96d6,1,0,...) at sleepq_wait+0x60
_cv_wait(c0c0f414,c0c0f3f4,c0aba083,1c2,0,...) at _cv_wait+0x1fc
audit_worker(0,e3c9cd38,c0a9920b,2ea,c5118000,...) at audit_worker+0x99
fork_exit(c091b170,0,e3c9cd38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe3c9cd70, ebp = 0 ---
Tracing command swapper pid 0 tid 0 td 0xc0bb0930
sched_switch(c0bb0930,0,1,180,4acff9e0,...) at sched_switch+0x49b
mi_switch(1,0,c0abe20a,2d9,c0ac912b,...) at mi_switch+0x217
scheduler(0,101ec00,101ec00,101e000,1028000,...) at scheduler+0x2f2
mi_startup() at mi_startup+0x96
begin() at begin+0x2c
db>
>How-To-Repeat:
Install RELENG_7 on i386/SMP
Let the box do something (compile KDE in my case)
Access the box remotely and do something, "ps ax" or cat /var/log/messages was it that triggered the faultr in this particular case.
In our case the IPSec link is over our firewall and I believe this is a key point.
>Fix:
>Release-Note:
>Audit-Trail:
>Unformatted:
More information about the freebsd-bugs
mailing list