ZFS DEADLKRES

Marcin Cieslak saper at saper.info
Mon Feb 22 00:24:37 UTC 2010




On Sat, 20 Feb 2010, Attilio Rao wrote:

> 2010/2/18 Marcin Cieslak <saper at saper.info>:
>> My r203753 amd64 laptop falls into the deadlock situation
>> every night while running periodic daily script. I am pretty
>> certain this is related to ZFS.
>>
>> I have enabled DEADLKRES in the kernel. I even have
>> a separate dump partition (not used for swap).
>
> May you reproduce the bug with WITNESS?
> If you can, you should enable DEADLKRES too and once it panics let
> extract a textdump(4) with the following commands:
> bt, ps, show alllocks, show pcpu, allthreads

Unfortunately, there is no way to write anything to disk.
All attempts to talk to the ata subsystem from ddb(4)
fail with EIO (probably timeout).

I did what I could using broken firewire console.

At first, "periodic daily" processing progresses normally
(lots of disk activity).

Before deadlock detector kicks in the system, although
busy, keeps responding to keystrokes. Any access
to the ata disk blocks the process. The hard drive
LED is on all the time (no blinking) and the drive
itself appears to be quiet.

Anyway, "ls" and "find" operations are usually
very slow (and generate lots of I/O activity).
This happens to me since I am running ZFS
(ca. FreeBSD  7.2-something).

Below the output requested (separated by dashes):

----------------
bt:

[dcons connected]

db> bt
Tracing pid 0 tid 100067 td 0xffffff00037b8390
kdb_enter() at kdb_enter+0x3d
panic() at panic+0x176
deadlkres() at deadlkres+0x229
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803dc5cd30, rbp = 0 ---

----------------
show pcpu:

db> show pcpu
cpuid        = 0
dynamic pcpu    = 0x14ced80
curthread    = 0xffffff00037b8390: pid 0 "deadlkres"
curpcb       = 0xffffff803dc5cd40
fpcurthread  = none
idlethread   = 0xffffff000336a390: pid 11 "idle: cpu0"
curpmap         = 0
tssp            = 0xffffffff80f3ec00
commontssp      = 0xffffffff80f3ec00
rsp0            = 0xffffff803dc5cd40
gs32p           = 0xffffffff80f3da38
ldt             = 0xffffffff80f3da78
tss             = 0xffffffff80f3da68
spin locks held:
db>

----------------
alllocks/vnodes:

db> shw ow alllocks
Process 3498 (sendmail) thread 0xffffff0009615390 (100158)
shared lockmgr zfs (zfs) r = 0 (0xffffff0006905098) locked @ /usr/src/sys/kern/vfs_subr.c:2091
Process 3496 (sendmail) thread 0xffffff004fbd1720 (100235)
shared lockmgr zfs (zfs) r = 0 (0xffffff0006905098) locked @ /usr/src/sys/kern/vfs_subr.c:2091
Process 3495 (sendmail) thread 0xffffff004fbd2ab0 (100226)
shared lockmgr zfs (zfs) r = 0 (0xffffff0079423098) locked @ /usr/src/sys/kern/vfs_syscalls.c:4102
Process 3494 (sendmail) thread 0xffffff004fbac390 (100224)
shared lockmgr zfs (zfs) r = 0 (0xffffff0006905098) locked @ /usr/src/sys/kern/vfs_subr.c:2091
Process 3207 (find) thread 0xffffff002c6fb000 (100190)
shared lockmgr zfs (zfs) r = 0 (0xffffff0014c38cf0) locked @ /usr/src/sys/kern/vfs_lookup.c:502
Process 2796 (login) thread 0xffffff000974e000 (100186)
shared lockmgr zfs (zfs) r = 0 (0xffffff00095e4588) locked @ /usr/src/sys/kern/vfs_subr.c:2091
Process 2560 (cron) thread 0xffffff002c69b720 (100196)
shared lockmgr zfs (zfs) r = 0 (0xffffff0076383588) locked @ /usr/src/sys/kern/vfs_lookup.c:502
Process 1991 (ntpd) thread 0xffffff002c64bab0 (100191)
exclusive lockmgr zfs (zfs) r = 0 (0xffffff0009541cf0) locked @ /usr/src/sys/kern/vfs_subr.c:2091
Process 1946 (hcsecd) thread 0xffffff0009753390 (100181)
exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xffffff002c6b3648) locked @ /usr/src/sys/kern/uipc_sockbuf.c:148
Process 1731 (amd) thread 0xffffff0009615000 (100159)
shared lockmgr zfs (zfs) r = 0 (0xffffff0006905098) locked @ /usr/src/sys/kern/vfs_subr.c:2091
Process 1590 (syslogd) thread 0xffffff0009618000 (100151)
exclusive lockmgr zfs (zfs) r = 0 (0xffffff001e7e8800) locked @ /usr/src/sys/kern/vfs_vnops.c:607
Process 23 (syncer) thread 0xffffff00069d3000 (100073)
exclusive lockmgr syncer (syncer) r = 0 (0xffffff00095e4800) locked @ /usr/src/sys/kern/vfs_subr.c:1701
db> show lockedvnods
Locked vnodes

0xffffff0006905000: tag zfs, type VDIR
     usecount 4, writecount 0, refcount 5 mountedhere 0
     flags ()
     v_object 0xffffff004fb97d80 ref 0 pages 0
     lock type zfs: SHARED (count 4)
#0 0xffffffff80603886 at __lockmgr_args+0x519
#1 0xffffffff80698b5a at vop_stdlock+0x39
#2 0xffffffff8096fb4f at VOP_LOCK1_APV+0x9b
#3 0xffffffff806b47cd at _vn_lock+0x5d
#4 0xffffffff806a8764 at vget+0x6c
#5 0xffffffff80696579 at cache_lookup+0x4dd
#6 0xffffffff806968bd at vfs_cache_lookup+0xad
#7 0xffffffff8097098b at VOP_LOOKUP_APV+0xba
#8 0xffffffff8069d1cf at lookup+0x45e
#9 0xffffffff8069e14a at namei+0x4cb
#10 0xffffffff806aca3c at kern_statat_vnhook+0x82
#11 0xffffffff806acbe1 at kern_statat+0x15
#xffffffff806acd36 at stat+0x22
#13 0xffffffff80922e9b at syscall+0x118
#14 0xffffffff8090a281 at Xfast_syscall+0xe1

0xffffff0014c38c58: tag zfs, type VDIR
     usecount 3, writecount 0, refcount 4 mountedhere 0
     flags ()
     v_object 0xffffff002936f948 ref 0 pages 0
     lock type zfs: SHARED (count 1)
#0 0xffffffff80603886 at __lockmgr_args+0x519
#1 0xffffffff80698b5a at vop_stdlock+0x39
#2 0xffffffff8096fb4f at VOP_LOCK1_APV+0x9b
#3 0xffffffff806b47cd at _vn_lock+0x5d
#4 0xffffffff8069ce5a at lookup+0xe9
#5 0xffffffff8069e14a at namei+0x4cb
#6 0xffffffff806aca3c at kern_statat_vnhook+0x82
#7 0xffffffff806acbe1 at kern_statat+0x15
#8 0xffffffff806acc7f at lstat+0x22
#9 0xffffffff80922e9b at syscall+0x118
#10 0xffffffff8090a281 at Xfast_syscall+0xe1

0xffffff00095e4768: tag syncer, type VNON
     usecount 1, writecount 0, refcount 2 mountedhere 0
     flags ()
     lock type syncer: EXCL by thread 0xffffff00069d3000 (pid 23)
#0 0xffffffff80603adb at __lockmgr_args+0x76e
#1 0xffffffff80698b5a at vop_stdlock+0x39
#2 0xffffffff8096fb4f at VOP_LOCK1_APV+0x9b
#3 0xffffffff806b47cd at _vn_lock+0x5d
#4 0xffffffff806a9ea9 at sync_vnode+0x11c
#5 0xffffffff806aa12b at sched_sync+0x1c6
#6 0xffffffff805f1f8a at fork_exit+0x12a
#7 0xffffffff8090a47e at fork_trampoline+0xe

0xffffff00095e44f0: tag zfs, type VDIR
     usecount 1, writecount 0, refcount 2 mountedhere 0
     flags ()
     v_object 0xffffff00094ef510 ref 0 pages 0
     lock type zfs: SHARED (count 1)
#0 0xffffffff80603886 at __lockmgr_args+0x519
#1 0xffffffff80698b5a at vop_stdlock+0x39
#2 0xffffffff8096fb4f at VOP_LOCK1_APV+0x9b
#3 0xffffffff806b47cd at _vn_lock+0x5d
#4 0xffffffff806a8764 at vget+0x6c
#5 0xffffffff80696579 at cache_lookup+0x4dd
#6 0xffffffff806968bd at vfs_cache_lookup+0xad
#7 0xffffffff8097098b at VOP_LOOKUP_APV+0xba
#8 0xffffffff8069d1cf at lookup+0x45e
#9 0xffffffff8069e14a at namei+0x4cb
#10 0xffffffff806b411b at vn_open_cred+0x1d7
#11 0xffffffff806b2519 at kern_openat+0x163
#12 0xffffffff80922e9b at syscall+0x118
#13 0xffffffff8090a281 at Xfast_syscall+0xe1

0xffffff0009541c58: tag zfs, type VDIR
     usecount 1, writecount 0, refcount 2 mountedhere 0
     flags ()
     lock type zfs: EXCL by thread 0xffffff002c64bab0 (pid 1991)
#0 0xffffffff80603886 at __lockmgr_args+0x519
#1 0xffffffff80698b5a at vop_stdlock+0x39
#2 0xffffffff8096fb4f at VOP_LOCK1_APV+0x9b
#3 0xffffffff806b47cd at _vn_lock+0x5d
#4 0xffffffff806a8764 at vget+0x6c
#5 0xffffffff80696579 at cache_lookup+0x4dd
#6 0xffffffff806968bd at vfs_cache_lookup+0xad
#7 0xffffffff8097098b at VOP_LOOKUP_APV+0xba
#8 0xffffffff8069d1cf at lookup+0x45e
#9 0xffffffff8069e14a at namei+0x4cb
#10 0xffffffff806b4006 at vn_open_cred+0xc2
#11 0xffffffff806b2519 at kern_openat+0x163
#12 0xffffffff80922e9b at syscall+0x118
#13 0xffffffff8090a281 at Xfast_syscall+0xe1

0xffffff0079423000: tag zfs, type VDIR
     usecount 9, writecount 0, refcount 10 mountedhere 0
     flags ()
     v_object 0xffffff002c6b8948 ref 0 pages 0
     lock type zfs: SHARED (count 1)
  with exclusive waiters pending
#0 0xffffffff80603886 at __lockmgr_args+0x519
#1 0xffffffff80698b5a at vop_stdlock+0x39
#2 0xffffffff8096fb4f at VOP_LOCK1_APV+0x9b
#3 0xffffffff806b47cd at _vn_lock+0x5d
#4 0xffffffff8069ce5a at lookup+0xe9
#5 0xffffffff8069e14a at namei+0x4cb
#6 0xffffffff806b411b at vn_open_cred+0x1d7
#7 0xffffffff806b2519 at kern_openat+0x163
#8 0xffffffff80922e9b at syscall+0x118
#9 0xffffffff8090a281 at Xfast_syscall+0xe1

0xffffff00763834f0: tag zfs, type VDIR
     usecount 2, writecount 0, refcount 2 mountedhere 0
     flags ()
     lock type zfs: SHARED (count 1)
#0 0xffffffff80603886 at __lockmgr_args+0x519
#1 0xffffffff80698b5a at vop_stdlock+0x39
#2 0xffffffff8096fb4f at VOP_LOCK1_APV+0x9b
#3 0xffffffff806b47cd at _vn_lock+0x5d
#4 0xffffffff8069ce5a at lookup+0xe9
#5 0xffffffff8069e14a at namei+0x4cb
#6 0xffffffff806aca3c at kern_statat_vnhook+0x82
#7 0xffffffff806acbe1 at kern_statat+0x15
#8 0xffffffff806acd36 at stat+0x22
#9 0xffffffff80922e9b at syscall+0x118
#10 0xffffffff8090a281 at Xfast_syscall+0xe1

0xffffff001e7e8768: tag zfs, type VREG
     usecount 1, writecount 1, refcount 1 mountedhere 0
     flags ()
     v_object 0xffffff002360e360 ref 0 pages 0
     lock type zfs: EXCL by thread 0xffffff0009618000 (pid 1590)
#0 0xffffffff80603adb at __lockmgr_args+0x76e
#1 0xffffffff80698b5a at vop_stdlock+0x39
#2 0xffffffff8096fb4f at VOP_LOCK1_APV+0x9b
#3 0xffffffff806b47cd at _vn_lock+0x5d
#4 0xffffffff806b5c8f at vn_write+0x1fb
#5 0xffffffff8065f5a9 at dofilewrite+0x7a
#6 0xffffffff80660c55 at kern_writev+0x52
#7 0xffffffff80660cd3 at writev+0x3a
#8 0xffffffff80922e9b at syscall+0x118
#9 0xffffffff8090a281 at Xfast_syscall+0xe1


----------------
attempt to execute minidump (this is from one of the previous
dumps, not the current one. The thread address pointed to
one of the cron threads that was locked @ /usr/src/sys/kern/vfs_subr.c:2091)

panic: deadlkres: possible deadlock detected for 0xffffff0006997000, blocked for 1800716 ticks

cpuid = 1
KDB: enter: panic
[thread pid 0 tid 100067]
Stopped at      kdb_enter+0x3d: movq    $0,0x6f923c(%rip)
db> call doadump()
Physical memeory: 2013 MB
Dumping 1784MB: Aborting dump due to I/O error.

** DUMP FAILED (ERROR 5) **
= 0
db>

----------------
attempt to execute full dump:

db> calld  doadump()
Dumping 2046 MB (2 chunks)
Aborting dump due to I/O error.

** DUMP FAILED (ERROR 5) **
= 0
db>

----------------
kernel config:

cpu		HAMMER
ident		VAIO

# To statically compile in device wiring instead of /boot/device.hints
#hints		"GENERIC.hints"		# Default places to look for devices.

# Use the following to compile in values accessible to the kernel
# through getenv() (or kenv(1) in userland). The format of the file
# is 'variable=value', see kenv(1)
#
# env		"GENERIC.env"

makeoptions	DEBUG=-g		# Build kernel with gdb(1) debug symbols

options 	SCHED_ULE		# ULE scheduler
options 	PREEMPTION		# Enable kernel thread preemption
options 	INET			# InterNETworking
options 	INET6			# IPv6 communications protocols
options		IPSEC			# IP security
options		IPSEC_NAT_T		# IP security NAT traversal
options		IPSEC_FILTERTUNNEL # Filter IPsec packets
options 	SCTP			# Stream Control Transmission Protocol
options 	FFS			# Berkeley Fast Filesystem
options 	SOFTUPDATES		# Enable FFS soft updates support
options 	UFS_ACL			# Support for access control lists
options 	UFS_DIRHASH		# Improve performance on big directories
options 	UFS_GJOURNAL		# Enable gjournal-based UFS journaling
options 	MD_ROOT			# MD is a potential root device
options 	NFSCL
options 	NFSCLIENT
options 	NFSSERVER
options 	NFSD
options 	NFSLOCKD		# Network Lock Manager
options		KGSSAPI
options 	NFS_ROOT		# NFS usable as /, requires NFSCLIENT
options 	NTFS			# NT File SYstem
options 	MSDOSFS			# MSDOS Filesystem
options 	CD9660			# ISO 9660 Filesystem
options 	PROCFS			# Process filesystem (requires PSEUDOFS)
options 	PSEUDOFS		# Pseudo-filesystem framework
options 	GEOM_PART_GPT		# GUID Partition Tables.
options 	GEOM_LABEL		# Provides labelization
options 	COMPAT_43TTY		# BSD 4.3 TTY compat (sgtty)
options 	COMPAT_IA32		# Compatible with i386 binaries
options 	COMPAT_FREEBSD4		# Compatible with FreeBSD4
options 	COMPAT_FREEBSD5		# Compatible with FreeBSD5
options 	COMPAT_FREEBSD6		# Compatible with FreeBSD6
options 	COMPAT_FREEBSD7		# Compatible with FreeBSD7
options 	SCSI_DELAY=5000		# Delay (in ms) before probing SCSI
options 	KTRACE			# ktrace(1) support
options 	STACK			# stack(9) support
options 	SYSVSHM			# SYSV-style shared memory
options 	SYSVMSG			# SYSV-style message queues
options 	SYSVSEM			# SYSV-style semaphores
options 	P1003_1B_SEMAPHORES	# POSIX-style semaphores
options 	_KPOSIX_PRIORITY_SCHEDULING # POSIX P1003_1B real-time extensions
options 	PRINTF_BUFR_SIZE=128	# Prevent printf output being interspersed.
options 	KBD_INSTALL_CDEV	# install a CDEV entry in /dev
options 	HWPMC_HOOKS		# Necessary kernel hooks for hwpmc(4)
options 	AUDIT			# Security event auditing
options 	MAC			# TrustedBSD MAC Framework
options		FLOWTABLE		# per-cpu routing cache
#options 	KDTRACE_FRAME		# Ensure frames are compiled in
#options 	KDTRACE_HOOKS		# Kernel DTrace hooks

# Debugging for use in -current
options 	KDB			# Enable kernel debugger support.
options 	DDB			# Support DDB.
options 	GDB			# Support remote GDB.
options 	INVARIANTS		# Enable calls of extra sanity checking
options 	INVARIANT_SUPPORT	# Extra sanity checks of internal structures, required by INVARIANTS
options 	WITNESS			# Enable checks to detect deadlocks and cycles
options 	WITNESS_SKIPSPIN	# Don't run witness on spinlocks for speed
options		DEADLKRES
options		CAMDEBUG
options		DEBUG_LOCKS
options		DEBUG_VFS_LOCKS
options		DIAGNOSTIC

# Make an SMP-capable kernel by default
options 	SMP			# Symmetric MultiProcessor Kernel

# CPU frequency control
device		cpufreq

# Bus support.
device		acpi
options     ACPI_DEBUG
device		pci

# Floppy drives
#device		fdc

# ATA and ATAPI devices
device		ata
device		atadisk		# ATA disk drives
device		ataraid		# ATA RAID drives
device		atapicd		# ATAPI CDROM drives
device		atapifd		# ATAPI floppy drives
device		atapist		# ATAPI tape drives
options 	ATA_STATIC_ID	# Static device numbering
device		ahci

#device		atacore
#device		atacard
#device		atapci
#device		ataahci
#device		ataintel

# SCSI Controllers
device		ahc		# AHA2940 and onboard AIC7xxx devices
options 	AHC_REG_PRETTY_PRINT	# Print register bitfields in debug
 					# output.  Adds ~128k to driver.
device		ahd		# AHA39320/29320 and onboard AIC79xx devices
options 	AHD_REG_PRETTY_PRINT	# Print register bitfields in debug
 					# output.  Adds ~215k to driver.
device		amd		# AMD 53C974 (Tekram DC-390(T))
device		hptiop		# Highpoint RocketRaid 3xxx series
device		isp		# Qlogic family
#device		ispfw		# Firmware for QLogic HBAs- normally a module
device		mpt		# LSI-Logic MPT-Fusion
#device		ncr		# NCR/Symbios Logic
device		sym		# NCR/Symbios Logic (newer chipsets + those of `ncr')
device		trm		# Tekram DC395U/UW/F DC315U adapters

device		adv		# Advansys SCSI adapters
device		adw		# Advansys wide SCSI adapters
device		aic		# Adaptec 15[012]x SCSI adapters, AIC-6[23]60.
device		bt		# Buslogic/Mylex MultiMaster SCSI adapters

# SCSI peripherals
device		scbus		# SCSI bus (required for SCSI)
device		ch		# SCSI media changers
device		da		# Direct Access (disks)
device		sa		# Sequential Access (tape etc)
device		cd		# CD
device		pass		# Passthrough device (direct SCSI access)
device		ses		# SCSI Environmental Services (and SAF-TE)

# RAID controllers interfaced to the SCSI subsystem
device		amr		# AMI MegaRAID
device		arcmsr		# Areca SATA II RAID
#XXX it is not 64-bit clean, -scottl
#device		asr		# DPT SmartRAID V, VI and Adaptec SCSI RAID
device		ciss		# Compaq Smart RAID 5*
device		dpt		# DPT Smartcache III, IV - See NOTES for options
device		hptmv		# Highpoint RocketRAID 182x
device		hptrr		# Highpoint RocketRAID 17xx, 22xx, 23xx, 25xx
device		iir		# Intel Integrated RAID
device		ips		# IBM (Adaptec) ServeRAID
device		mly		# Mylex AcceleRAID/eXtremeRAID
device		twa		# 3ware 9000 series PATA/SATA RAID

# RAID controllers
device		aac		# Adaptec FSA RAID
device		aacp		# SCSI passthrough for aac (requires CAM)
device		ida		# Compaq Smart RAID
device		mfi		# LSI MegaRAID SAS
device		mlx		# Mylex DAC960 family
#XXX pointer/int warnings
#device		pst		# Promise Supertrak SX6000
device		twe		# 3ware ATA RAID

# atkbdc0 controls both the keyboard and the PS/2 mouse
device		atkbdc		# AT keyboard controller
device		atkbd		# AT keyboard
device		psm		# PS/2 mouse

device		kbdmux		# keyboard multiplexer

device		vga		# VGA video card driver

device		splash		# Splash screen and screen saver support

# syscons is the default console driver, resembling an SCO console
device		sc

device		agp		# support several AGP chipsets

# PCCARD (PCMCIA) support
# PCMCIA and cardbus bridge support
device		cbb		# cardbus (yenta) bridge
device		pccard		# PC Card (16-bit) bus
device		cardbus		# CardBus (32-bit) bus

# Serial (COM) ports
device		uart		# Generic UART driver

# Parallel port
device		ppc
device		ppbus		# Parallel port bus (required)
device		lpt		# Printer
device		plip		# TCP/IP over parallel
device		ppi		# Parallel port interface device
#device		vpo		# Requires scbus and da

# If you've got a "dumb" serial or parallel PCI card that is
# supported by the puc(4) glue driver, uncomment the following
# line to enable it (connects to sio, uart and/or ppc drivers):
#device		puc

# PCI Ethernet NICs.
device		de		# DEC/Intel DC21x4x (``Tulip'')
device		em		# Intel PRO/1000 Gigabit Ethernet Family
device		igb		# Intel PRO/1000 PCIE Server Gigabit Family
device		ixgbe		# Intel PRO/10GbE PCIE Ethernet Family
device		le		# AMD Am7900 LANCE and Am79C9xx PCnet
device		ti		# Alteon Networks Tigon I/II gigabit Ethernet
device		txp		# 3Com 3cR990 (``Typhoon'')
device		vx		# 3Com 3c590, 3c595 (``Vortex'')

# PCI Ethernet NICs that use the common MII bus controller code.
# NOTE: Be sure to keep the 'device miibus' line in order to use these NICs!
device		miibus		# MII bus support
device		ae		# Attansic/Atheros L2 FastEthernet
device		age		# Attansic/Atheros L1 Gigabit Ethernet
device		alc		# Atheros AR8131/AR8132 Ethernet
device		ale		# Atheros AR8121/AR8113/AR8114 Ethernet
device		bce		# Broadcom BCM5706/BCM5708 Gigabit Ethernet
device		bfe		# Broadcom BCM440x 10/100 Ethernet
device		bge		# Broadcom BCM570xx Gigabit Ethernet
device		dc		# DEC/Intel 21143 and various workalikes
device		et		# Agere ET1310 10/100/Gigabit Ethernet
device		fxp		# Intel EtherExpress PRO/100B (82557, 82558)
device		jme		# JMicron JMC250 Gigabit/JMC260 Fast Ethernet
device		lge		# Level 1 LXT1001 gigabit Ethernet
device		msk		# Marvell/SysKonnect Yukon II Gigabit Ethernet
device		nfe		# nVidia nForce MCP on-board Ethernet
device		nge		# NatSemi DP83820 gigabit Ethernet
#device		nve		# nVidia nForce MCP on-board Ethernet Networking
device		pcn		# AMD Am79C97x PCI 10/100 (precedence over 'le')
device		re		# RealTek 8139C+/8169/8169S/8110S
device		rl		# RealTek 8129/8139
device		sf		# Adaptec AIC-6915 (``Starfire'')
device		sis		# Silicon Integrated Systems SiS 900/SiS 7016
device		sk		# SysKonnect SK-984x & SK-982x gigabit Ethernet
device		ste		# Sundance ST201 (D-Link DFE-550TX)
device		stge		# Sundance/Tamarack TC9021 gigabit Ethernet
device		tl		# Texas Instruments ThunderLAN
device		tx		# SMC EtherPower II (83c170 ``EPIC'')
device		vge		# VIA VT612x gigabit Ethernet
device		vr		# VIA Rhine, Rhine II
device		wb		# Winbond W89C840F
device		xl		# 3Com 3c90x (``Boomerang'', ``Cyclone'')

# ISA Ethernet NICs.  pccard NICs included.
device		cs		# Crystal Semiconductor CS89x0 NIC
# 'device ed' requires 'device miibus'
device		ed		# NE[12]000, SMC Ultra, 3c503, DS8390 cards
device		ex		# Intel EtherExpress Pro/10 and Pro/10+
device		ep		# Etherlink III based cards
device		fe		# Fujitsu MB8696x based cards
device		sn		# SMC's 9000 series of Ethernet chips
device		xe		# Xircom pccard Ethernet

# Wireless NIC cards
device		wlan		# 802.11 support
options 	IEEE80211_DEBUG	# enable debug msgs
options 	IEEE80211_AMPDU_AGE # age frames in AMPDU reorder q's
options 	IEEE80211_SUPPORT_MESH	# enable 802.11s draft support
device		wlan_wep	# 802.11 WEP support
device		wlan_ccmp	# 802.11 CCMP support
device		wlan_tkip	# 802.11 TKIP support
device		wlan_amrr	# AMRR transmit rate control algorithm
device		an		# Aironet 4500/4800 802.11 wireless NICs.
device		ath		# Atheros pci/cardbus NIC's
device		ath_hal		# pci/cardbus chip support
options 	AH_SUPPORT_AR5416	# enable AR5416 tx/rx descriptors
device		ath_rate_sample	# SampleRate tx rate control for ath
device		ral		# Ralink Technology RT2500 wireless NICs.
device		wi		# WaveLAN/Intersil/Symbol 802.11 wireless NICs.

# Pseudo devices.
device		loop		# Network loopback
device		random		# Entropy device
device		ether		# Ethernet support
device		tun		# Packet tunnel.
device		pty		# BSD-style compatibility pseudo ttys
device		md		# Memory "disks"
device		gif		# IPv6 and IPv4 tunneling
device		faith		# IPv6-to-IPv4 relaying (translation)
device		firmware	# firmware assist module
device		crypto		# for IP security

# The `bpf' device enables the Berkeley Packet Filter.
# Be aware of the administrative consequences of enabling this!
# Note that 'bpf' is required for DHCP.
device		bpf		# Berkeley packet filter

# USB support
device		uhci		# UHCI PCI->USB interface
device		ohci		# OHCI PCI->USB interface
device		ehci		# EHCI PCI->USB interface (USB 2.0)
device		usb		# USB Bus (required)
options     USB_DEBUG
#device		udbp		# USB Double Bulk Pipe devices
device		uhid		# "Human Interface Devices"
device		ukbd		# Keyboard
device		ulpt		# Printer
device		umass		# Disks/Mass storage - Requires scbus and da
device		ums		# Mouse
device		urio		# Diamond Rio 500 MP3 player
# USB Serial devices
device		uark		# Technologies ARK3116 based serial adapters
device		ubsa		# Belkin F5U103 and compatible serial adapters
device		uftdi		# For FTDI usb serial adapters
device		uipaq		# Some WinCE based devices
device		uplcom		# Prolific PL-2303 serial adapters
device		uslcom		# SI Labs CP2101/CP2102 serial adapters
device		uvisor		# Visor and Palm devices
device		uvscom		# USB serial support for DDI pocket's PHS
# USB Ethernet, requires miibus
device		aue		# ADMtek USB Ethernet
device		axe		# ASIX Electronics USB Ethernet
device		cdce		# Generic USB over Ethernet
device		cue		# CATC USB Ethernet
device		kue		# Kawasaki LSI USB Ethernet
device		rue		# RealTek RTL8150 USB Ethernet
device		udav		# Davicom DM9601E USB
# USB Wireless
device		rum		# Ralink Technology RT2501USB wireless NICs
device		uath		# Atheros AR5523 wireless NICs
device		ural		# Ralink Technology RT2500USB wireless NICs
device		zyd		# ZyDAS zb1211/zb1211b wireless NICs

# FireWire support
device		firewire	# FireWire bus code
device		sbp		# SCSI over FireWire (Requires scbus and da)
device		fwe		# Ethernet over FireWire (non-standard!)
device		fwip		# IP over FireWire (RFC 2734,3146)
device		dcons		# Dumb console driver
device		dcons_crom	# Configuration ROM for dcons

----------------
dmesg since boot:

[dcons connected]
Consoles: internal video/keyboard  dumb console port 
BIOS drive C: is disk0
BIOS drive D: is disk1
BIOS 638kB/2094592kB available memory

FreeBSD/i386 bootstrap loader, Revision 1.1
(saper at radziecki.saper.info, Wed Jan 20 22:26:53 CET 2010)
|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\Loading /boot/defaults/loader.conf 
|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\/boot/kernel/kernel text=0x9acf28 |/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/!
 -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\data=0x13ac18+0x279ac0 |/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-syms=[0x8+0xfbd60\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/+0x8+0xe35e5-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/!
 -\|/-\|/-]
\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/boot/kernel/zfs.ko /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-size 0x18ee60 at 0x1141000
loading required module 'opensolaris'
\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/boot/kernel/opensolaris.ko size 0x38a8 at 0x12d0000
/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/boot/kernel/linux.ko /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/size 0x431f8 at 0x12d4000
-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\/boot/kernel/if_wpi.ko |/-\|/-\|/-\|/-\|/-\|size 0x12678 at 0x1318000
/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-/boot/kernel/snd_hda.ko \|/-\|/-\|/-\|/-\|/-\|/-\|/-size 0x24668 at 0x132b000
loading required module 'sound'
\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/boot/kernel/sound.ko /-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/size 0x86030 at 0x1350000
-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|//boot/kernel/acpi_video.ko -\|/-\|/-\|/size 0x6968 at 0x13d7000
-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/boot/kernel/wpifw.ko /-\|/-\|/-\|/-\|/size 0x25588 at 0x13de000
-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|//boot/kernel/acpi_sony.ko size 0x1fb0 at 0x1404000
-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|//boot/kernel/smbus.ko size 0x2498 at 0x1406000
-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|//boot/kernel/smb.ko size 0x2dc8 at 0x1409000
-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|//boot/kernel/iicsmb.ko size 0x3cf0 at 0x140c000
loading required module 'iicbus'
-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|//boot/kernel/iicbus.ko size 0x43b0 at 0x1410000
-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|//boot/kernel/ichsmb.ko -\|/-\|/-\size 0x59c8 at 0x1415000
|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\/boot/kernel/ng_ubt.ko |/-\|/-\|/-\|size 0x82c8 at 0x141b000
loading required module 'netgraph'
/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/boot/kernel/netgraph.ko /-\|/-\|/-\|/-\|/-\|/-\|/-\size 0x15b48 at 0x1424000
loading required module 'ng_hci'
|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\/boot/kernel/ng_hci.ko |/-\|/-\|/-\|/-\|/-\|size 0x12f08 at 0x143a000
loading required module 'ng_bluetooth'
/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/boot/kernel/ng_bluetooth.ko size 0x2d20 at 0x144d000
/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/boot/kernel/atapicam.ko /-\|/-\|/-\|/-size 0x6de8 at 0x1450000
\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-/boot/kernel/cmx.ko \|/-\|/-size 0x4438 at 0x1457000
\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\/boot/modules/nvidia.ko |/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/!
 -\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\!
 |/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-!
 \|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|/-\|size 0xd3d630 at 0x145c000
/-\|/-\|/-\|/-\|/-\|/
Hit [Enter] to boot immediately, or any other key for command prompt.
 
Booting [/boot/kernel/kernel]... 
cryptosoft0: <software crypto> on motherboard
acpi0: <Sony> on motherboard
acpi0: [ITHREAD]
acpi_hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 900
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0
acpi_ec0: <Embedded Controller: GPE 0x17> port 0x62,0x66 on acpi0
acpi_lid0: <Control Method Lid Switch> on acpi0
acpi_button0: <Power Button> on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> irq 16 at device 1.0 on pci0
pci1: <ACPI PCI bus> on pcib1
vgapci0: <VGA-compatible display> mem 0xcd000000-0xcdffffff,0xd0000000-0xdfffffff,0xce000000-0xceffffff irq 16 at device 0.0 on pci1
nvidia0: <GeForce Go 7400> on vgapci0
vgapci0: child nvidia0 requested pci_enable_busmaster
vgapci0: child nvidia0 requested pci_enable_io
vgapci0: child nvidia0 requested pci_enable_io
nvidia0: [GIANT-LOCKED]
nvidia0: [ITHREAD]
hdac0: <Intel 82801G High Definition Audio Controller> mem 0xf8500000-0xf8503fff irq 21 at device 27.0 on pci0
hdac0: HDA Driver Revision: 20100122_0141
hdac0: [ITHREAD]
pcib2: <ACPI PCI-PCI bridge> irq 16 at device 28.0 on pci0
pci2: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> irq 17 at device 28.1 on pci0
pci6: <ACPI PCI bus> on pcib3
wpi0: <Intel(R) PRO/Wireless 3945ABG> mem 0xf8100000-0xf8100fff irq 17 at device 0.0 on pci6
wpi0: [ITHREAD]
pcib4: <ACPI PCI-PCI bridge> irq 18 at device 28.2 on pci0
pci7: <ACPI PCI bus> on pcib4
mskc0: <Marvell Yukon 88E8036 Fast Ethernet> port 0x3000-0x30ff mem 0xf8000000-0xf8003fff irq 18 at device 0.0 on pci7
msk0: <Marvell Technology Group Ltd. Yukon FE Id 0xb7 Rev 0x01> on mskc0
msk0: disabling jumbo frame support
msk0: Ethernet address: 00:1a:80:55:25:5e
miibus0: <MII bus> on msk0
e1000phy0: <Marvell 88E3082 10/100 Fast Ethernet PHY> PHY 0 on miibus0
e1000phy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
mskc0: [FILTER]
pcib5: <ACPI PCI-PCI bridge> irq 19 at device 28.3 on pci0
pci8: <ACPI PCI bus> on pcib5
uhci0: <Intel 82801G (ICH7) USB controller USB-A> port 0x1800-0x181f irq 19 at device 29.0 on pci0
uhci0: [ITHREAD]
usbus0: <Intel 82801G (ICH7) USB controller USB-A> on uhci0
uhci1: <Intel 82801G (ICH7) USB controller USB-B> port 0x1820-0x183f irq 19 at device 29.1 on pci0
uhci1: [ITHREAD]
usbus1: <Intel 82801G (ICH7) USB controller USB-B> on uhci1
uhci2: <Intel 82801G (ICH7) USB controller USB-C> port 0x1840-0x185f irq 19 at device 29.2 on pci0
uhci2: [ITHREAD]
usbus2: <Intel 82801G (ICH7) USB controller USB-C> on uhci2
uhci3: <Intel 82801G (ICH7) USB controller USB-D> port 0x1860-0x187f irq 19 at device 29.3 on pci0
uhci3: [ITHREAD]
usbus3: <Intel 82801G (ICH7) USB controller USB-D> on uhci3
ehci0: <Intel 82801GB/R (ICH7) USB 2.0 controller> mem 0xf8504000-0xf85043ff irq 23 at device 29.7 on pci0
ehci0: [ITHREAD]
usbus4: EHCI version 1.0
usbus4: <Intel 82801GB/R (ICH7) USB 2.0 controller> on ehci0
pcib6: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci9: <ACPI PCI bus> on pcib6
cbb0: <PCI-CardBus Bridge> at device 4.0 on pci9
cardbus0: <CardBus bus> on cbb0
pccard0: <16-bit PCCard bus> on cbb0
cbb0: [FILTER]
fwohci0: <1394 Open Host Controller Interface> mem 0xf8205000-0xf82057ff,0xf8200000-0xf8203fff irq 21 at device 4.1 on pci9
fwohci0: [ITHREAD]
fwohci0: OHCI version 1.10 (ROM=1)
fwohci0: No. of Isochronous channels is 4.
fwohci0: EUI64 08:00:46:03:02:91:b4:4d
fwohci0: Phy 1394a available S400, 2 ports.
fwohci0: Link S400, max_rec 2048 bytes.
firewire0: <IEEE1394(FireWire) bus> on fwohci0
dcons_crom0: <dcons configuration ROM> on firewire0
dcons_crom0: bus_addr 0x41000
fwe0: <Ethernet over FireWire> on firewire0
if_fwe0: Fake Ethernet address: 0a:00:46:91:b4:4d
fwe0: Ethernet address: 0a:00:46:91:b4:4d
fwip0: <IP over FireWire> on firewire0
fwip0: Firewire address: 08:00:46:03:02:91:b4:4d @ 0xfffe00000000, S400, maxrec 2048
sbp0: <SBP-2/SCSI over FireWire> on firewire0
fwohci0: Initiate bus reset
fwohci0: fwohci_intr_core: BUS reset
fwohci0: fwohci_intr_core: node_id=0x00000000, SelfID Count=1, CYCLEMASTER mode
pci9: <mass storage> at device 4.2 (no driver attached)
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel ICH7 UDMA100 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1880-0x188f at device 31.1 on pci0
ata0: <ATA channel 0> on atapci0
ata0: [ITHREAD]
ahci0: <Intel ICH7M AHCI SATA controller> port 0x18c8-0x18cf,0x18ac-0x18af,0x18c0-0x18c7,0x18a8-0x18ab,0x18b0-0x18bf mem 0xf8504400-0xf85047ff irq 22 at device 31.2 on pci0
ahci0: [ITHREAD]
ahci0: AHCI v1.10 with 4 1.5Gbps ports, Port Multiplier not supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich0: [ITHREAD]
ahcich1: <AHCI channel> at channel 2 on ahci0
ahcich1: [ITHREAD]
ichsmb0: <Intel 82801GB (ICH7) SMBus controller> port 0x18e0-0x18ff at device 31.3 on pci0
ichsmb0: can't get IRQ
device_attach: ichsmb0 attach returned 6
acpi_tz0: <Thermal Zone> on acpi0
acpi_tz1: <Thermal Zone> on acpi0
acpi_tz2: <Thermal Zone> on acpi0
atrtc0: <AT realtime clock> port 0x70-0x77 irq 8 on acpi0
atrtc0: Warning: Couldn't map I/O.
acpi_sony0: <Sony notebook controller> on acpi0
acpi_sony0: PID 0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
atkbd0: [ITHREAD]
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: [ITHREAD]
psm0: model GlidePoint, device ID 0
battery0: <ACPI Control Method Battery> on acpi0
acpi_acad0: <AC Adapter> on acpi0
cpu0: <ACPI CPU> on acpi0
est0: <Enhanced SpeedStep Frequency Control> on cpu0
p4tcc0: <CPU Frequency Thermal Control> on cpu0
cpu1: <ACPI CPU> on acpi0
est1: <Enhanced SpeedStep Frequency Control> on cpu1
p4tcc1: <CPU Frequency Thermal Control> on cpu1
orm0: <ISA Option ROMs> at iomem 0xc0000-0xcefff,0xdc000-0xdffff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
ppc0: cannot reserve I/O port range
ZFS NOTICE: Prefetch is disabled by default if less than 4GB of RAM is present;
             to enable, add "vfs.zfs.prefetch_disable=0" to /boot/loader.conf.
ZFS filesystem version 3
ZFS storage pool version 14
Timecounters tick every 1.000 msec
IPsec: Initialized Security Association Processing.
firewire0: 1 nodes, maxhop <= 0 cable IRM irm(0)  (me) 
firewire0: bus manager 0 
usbus0: 12Mbps Full Speed USB v1.0
usbus1: 12Mbps Full Speed USB v1.0
usbus2: 12Mbps Full Speed USB v1.0
usbus3: 12Mbps Full Speed USB v1.0
usbus4: 480Mbps High Speed USB v2.0
battery0: battery initialization start
acpi_acad0: acline initialization start
acpi_acad0: On Line
acpi_acad0: acline initialization done, tried 1 times
ugen0.1: <Intel> at usbus0
uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
ugen1.1: <Intel> at usbus1
uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
ugen2.1: <Intel> at usbus2
uhub2: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2
ugen3.1: <Intel> at usbus3
uhub3: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus3
ugen4.1: <Intel> at usbus4
uhub4: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus4
acd0: DVDR <MATSHITADVD-RAM UJ-852S/1.01> at ata0-master UDMA33 
hdac0: HDA Codec #0: Sigmatel STAC9872AK
hdac0: HDA Codec #1: Conexant (Unknown)
pcm0: <HDA Sigmatel STAC9872AK PCM #0 Analog> at cad 0 nid 1 on hdac0
pcm1: <HDA Sigmatel STAC9872AK PCM #1 Analog> at cad 0 nid 1 on hdac0
cmx0: <OMNIKEY CardMan 4040> at port 0x100-0x107 irq 20 function 0 config 35 on pccard0
cmx0: attached
Expensive timeout(9) function: 0xffffffff80653cff(0xffffff0003757000) 0.036358531 s
uhub0: 2 ports with 2 removable, self powered
uhub1: 2 ports with 2 removable, self powered
uhub2: 2 ports with 2 removable, self powered
uhub3: 2 ports with 2 removable, self powered
acd0: FAILURE - READ_BIG ILLEGAL REQUEST asc=0x64 ascq=0x00 
acd0: FAILURE - READ_BIG ILLEGAL REQUEST asc=0x64 ascq=0x00 
uhub4: 8 ports with 8 removable, self powered
acd0: FAILURE - INQUIRY ILLEGAL REQUEST asc=0x24 ascq=0x00 
ugen4.2: <USBest Technology> at usbus4
umass0: <USBest Technology USB Mass Storage Device, class 0/0, rev 2.00/1.00, addr 2> on usbus4
umass0:  SCSI over Bulk-Only; quirks = 0x0000
ada0 at ahcich0 bus 0 scbus1 target 0 lun 0
ada0: <FUJITSU MHW2120BH 00000012> ATA-8cd0 at ata0 bus 0 scbus3 target 0 lun 0
cd0: <MATSHITA DVD-RAM UJ-852S 1.01> Removable CD-ROM SCSI-0 device 
cd0: 33.000MB/s transfers
cd0: cd present [64816 x 2048 byte records]
  SATA 1.x device
ada0: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes)
ada0: Command Queueing enabled
ada0: 114473MB (234441648 512 byte sectors: 16H 63S/T 16383C)
SMP: AP CPU #1 Launched!
WARNING: WITNESS option enabled, expect reduced performance.
WARNING: DIAGNOSTIC option enabled, expect reduced performance.
acd0: FAILURE - READ_BIG ILLEGAL REQUEST asc=0x64 ascq=0x00 
(cd0:ata0:0:0:0): READ(10). CDB: 28 0 0 0 fd 2f 0 0 1 0 
(cd0:ata0:0:0:0): CAM status: SCSI Status Error
(cd0:ata0:0:0:0): SCSI status: Check Condition
(cd0:ata0:0:0:0): SCSI sense: ILLEGAL REQUEST csi:0,1d,0,0 asc:64,0 (Illegal mode for this track)
(cd0:ata0:0:0:0): cddone: got error 0x6 back
umass0:4:0:-1: Attached to scbus4
da0 at umass-sim0 bus 0 scbus4 target 0 lun 0
da0: <Ut163 USB2FlashStorage 0.00> Removable Direct Access SCSI-2 device 
da0: 40.000MB/s transfers
da0: 250MB (512000 512 byte sectors: 64H 32S/T 250C)
GEOM: ada0: partition 3 does not start on a track boundary.
GEOM: ada0: partition 3 does not end on a track boundary.
GEOM: ada0: partition 2 does not start on a track boundary.
GEOM: ada0: partition 2 does not end on a track boundary.
ugen4.3: <Sony> at usbus4
umass1: <Sony UMH-U09, class 0/0, rev 2.00/4.82, addr 3> on usbus4
umass1:  SCSI over Bulk-Only; quirks = 0x0000
GEOM: da0: partition 1 does not start on a track boundary.
GEOM: da0: partition 1 does not end on a track boundary.
Root mount waiting for: usbus4
umass1:5:1:-1: Attached to scbus5
Root mount waiting for: usbus4
(probe0:umass-sim1:1:0:0): TEST UNIT READY. CDB: 0 0 0 0 0 0 
(probe0:umass-sim1:1:0:0): CAM status: SCSI Status Error
(probe0:umass-sim1:1:0:0): SCSI status: Check Condition
(probe0:umass-sim1:1:0:0): SCSI sense: NOT READY asc:3a,0 (Medium not present)
da1 at umass-sim1 bus 1 scbus5 target 0 lun 0
da1: <Sony USB   HS-CARD 4.82> Removable Direct Access SCSI-0 device 
da1: 40.000MB/s transfers
da1: Attempt to query device size failed: NOT READY, Medium not present
ugen4.4: <vendor 0x05ca> at usbus4
Trying to mount root from zfs:tank
ugen3.2: <ALPS> at usbus3
ubt0: <ALPS UGX, class 224/1, rev 2.00/19.15, addr 2> on usbus3
WARNING: /crash was not properly dismounted
wlan0: Ethernet address: 00:1b:77:bc:a5:44
wpi0: need multicast update callback
msk0: link state changed to DOWN
WARNING: attempt to domain_add(bluetooth) after domainfinalize()
WARNING: attempt to domain_add(netgraph) after domainfinalize()
lock order reversal:
  1st 0xffffff80003ad018 wpi0_com_lock (wpi0_com_lock) @ /usr/src/sys/modules/wpi/../../dev/wpi/if_wpi.c:1292
  2nd 0xffffff00034f5010 wpi0 (network driver) @ /usr/src/sys/modules/wpi/../../dev/wpi/if_wpi.c:2069
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
_witness_debugger() at _witness_debugger+0x49
witness_checkorder() at witness_checkorder+0x7ea
_mtx_lock_flags() at _mtx_lock_flags+0x68
wpi_raw_xmit() at wpi_raw_xmit+0x68
ieee80211_send_mgmt() at ieee80211_send_mgmt+0x49c
sta_newstate() at sta_newstate+0x42b
wpi_newstate() at wpi_newstate+0xe3
ieee80211_newstate_cb() at ieee80211_newstate_cb+0xac
taskqueue_run() at taskqueue_run+0x90
taskqueue_thread_loop() at taskqueue_thread_loop+0x3f
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8000183d30, rbp = 0 ---
wlan0: link state changed to UP
wpi0: need multicast update callback
wpi0: need multicast update callback
wpi0: need multicast update callback
ipfw2 (+ipv6) initialized, divert loadable, nat loadable, rule-based forwarding disabled, default to deny, logging disabled
lock order reversal:
  1st 0xffffff0009adf310 zfs (zfs) @ /usr/src/sys/kern/vfs_mount.c:1058
  2nd 0xffffff0009af8cf0 devfs (devfs) @ /usr/src/sys/kern/vfs_subr.c:2091
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
_witness_debugger() at _witness_debugger+0x49
witness_checkorder() at witness_checkorder+0x7ea
__lockmgr_args() at __lockmgr_args+0xd5c
vop_stdlock() at vop_stdlock+0x39
VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b
_vn_lock() at _vn_lock+0x5d
vget() at vget+0x6c
devfs_allocv() at devfs_allocv+0xee
devfs_root() at devfs_root+0x41
vfs_donmount() at vfs_donmount+0xfa8
nmount() at nmount+0x74
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (378, FreeBSD ELF64, nmount), rip = 0x8007ac37c, rsp = 0x7fffffffdd28, rbp = 0x800a06048 ---
wpi0: need multicast update callback
battery0: battery initialization failed, giving up
kqemu version 0x00010400
kqemu: KQEMU installed, max_locked_mem=1030772kB.

FreeBSD/amd64 (radziecki.saper.info) (dcons)

login: fwohci0: fwohci_intr_core: BUS reset
fwohci0: fwohci_intr_core: node_id=0x00000000, SelfID Count=2, non CYCLEMASTER mode
firewire0: 2 nodes, maxhop <= 1 cable IRM irm(1) 
firewire0: fw_explore_node: Pre 1394a-2000 detected
firewire0: New S400 device ID:001106005350d195


FreeBSD/amd64 (radziecki.saper.info) (dcons)

login:

FreeBSD/amd64 (radziecki.saper.info) (dcons)

login: acpi_lid0: Lid closed
fwohci0: fwohci_intr_core: BUS reset
fwohci0: fwohci_intr_core: node_id=0x00000000, SelfID Count=3, non CYCLEMASTER mode
firewire0: 2 nodes, maxhop <= 1 cable IRM irm(1) 
fwohci0: fwohci_intr_core: BUS reset
fwohci0: fwohci_intr_core: node_id=0x00000000, SelfID Count=4, non CYCLEMASTER mode
firewire0: 2 nodes, maxhop <= 1 cable IRM irm(1) 
fwohci0: fwohci_intr_core: BUS reset
fwohci0: fwohci_intr_core: node_id=0x00000000, SelfID Count=5, non CYCLEMASTER mode
firewire0: 2 nodes, maxhop <= 1 cable IRM irm(1) 
fwohci0: fwohci_intr_core: BUS reset
fwohci0: fwohci_intr_core: node_id=0x00000000, SelfID Count=6, CYCLEMASTER mode
firewire0: 1 nodes, maxhop <= 0 cable IRM irm(0)  (me) 
firewire0: bus manager 0 
firewire0: fw_attach_dev:Removing missing device ID:001106005350d195
fwohci0: fwohci_intr_core: BUS reset
fwohci0: fwohci_intr_core: node_id=0x00000001, SelfID Count=7, CYCLEMASTER mode
firewire0: 2 nodes, maxhop <= 1 cable IRM irm(1)  (me) 
firewire0: bus manager 1 
firewire0: fw_explore_node: Pre 1394a-2000 detected
firewire0: New S400 device ID:001106005350d195
fwohci0: fwohci_intr_core: BUS reset
fwohci0: fwohci_intr_core: node_id=0x00000000, SelfID Count=8, non CYCLEMASTER mode
firewire0: 2 nodes, maxhop <= 1 cable IRM irm(1) 
fwohci0: too many cycle lost, no cycle master presents?


FreeBSD/amd64 (radziecki.saper.info) (dcons)

login: fwohci0: fwohci_intr_core: BUS reset
fwohci0: fwohci_intr_core: node_id=0x00000000, SelfID Count=9, non CYCLEMASTER mode
firewire0: 2 nodes, maxhop <= 1 cable IRM irm(1) 
fwohci0: too many cycle lost, no cycle master presents?
acpi_lid0: Lid opened
acpi_lid0: Lid closed
acpi_lid0: Lid opened
~.

FreeBSD/amd64 (radziecki.saper.info) (dcons)

login:

FreeBSD/amd64 (radziecki.saper.info) (dcons)

login:

FreeBSD/amd64 (radziecki.saper.info) (dcons)

login:

FreeBSD/amd64 (radziecki.saper.info) (dcons)

login:

FreeBSD/amd64 (radziecki.saper.info) (dcons)

login: ~.[dcons disconnected (read buffer failed)]

----------------
alltrace:


[dcons connected]


Tracing command sendmail pid 3499 tid 100166 td 0xffffff0009747390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
__lockmgr_args() at __lockmgr_args+0x77e
vop_stdlock() at vop_stdlock+0x39
VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b
_vn_lock() at _vn_lock+0x5d
cache_lookup() at cache_lookup+0x6a0
vfs_cache_lookup() at vfs_cache_lookup+0xad
VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xba
lookup() at lookup+0x45e
namei() at namei+0x4cb
vn_open_cred() at vn_open_cred+0x1d7
kern_openat() at kern_openat+0x163
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (5, FreeBSD ELF64, open), rip = 0x800ec012c, rsp = 0x7fffffffc238, rbp = 0x3 ---

Tracing command sendmail pid 3498 tid 100158 td 0xffffff0009615390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_cv_wait() at _cv_wait+0x15d
dbuf_read() at dbuf_read+0x34e
dmu_buf_hold() at dmu_buf_hold+0xb7
zap_lockdir() at zap_lockdir+0x48
zap_lookup_norm() at zap_lookup_norm+0x3a
zap_lookup() at zap_lookup+0x2e
zfs_dirent_lock() at zfs_dirent_lock+0x4f8
zfs_dirlook() at zfs_dirlook+0x5a
zfs_lookup() at zfs_lookup+0x1de
zfs_freebsd_lookup() at zfs_freebsd_lookup+0x7a
VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0xb1
vfs_cache_lookup() at vfs_cache_lookup+0xda
VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xba
lookup() at lookup+0x45e
namei() at namei+0x4cb
kern_statat_vnhook() at kern_statat_vnhook+0x82
kern_statat() at kern_statat+0x15
stat() at stat+0x22
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (188, FreeBSD ELF64, stat), rip = 0x800ec00cc, rsp = 0x7fffffffbec8, rbp = 0 ---

Tracing command sendmail pid 3497 tid 100183 td 0xffffff000974eab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
__lockmgr_args() at __lockmgr_args+0x77e
vop_stdlock() at vop_stdlock+0x39
VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b
_vn_lock() at _vn_lock+0x5d
cache_lookup() at cache_lookup+0x6a0
vfs_cache_lookup() at vfs_cache_lookup+0xad
VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xba
lookup() at lookup+0x45e
namei() at namei+0x4cb
vn_open_cred() at vn_open_cred+0x1d7
kern_openat() at kern_openat+0x163
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (5, FreeBSD ELF64, open), rip = 0x800ec012c, rsp = 0x7fffffffc238, rbp = 0x3 ---

Tracing command sendmail pid 3496 tid 100235 td 0xffffff004fbd1720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_cv_wait() at _cv_wait+0x15d
dbuf_read() at dbuf_read+0x34e
dmu_buf_hold() at dmu_buf_hold+0xb7
zap_lockdir() at zap_lockdir+0x48
zap_lookup_norm() at zap_lookup_norm+0x3a
zap_lookup() at zap_lookup+0x2e
zfs_dirent_lock() at zfs_dirent_lock+0x4f8
zfs_dirlook() at zfs_dirlook+0x5a
zfs_lookup() at zfs_lookup+0x1de
zfs_freebsd_lookup() at zfs_freebsd_lookup+0x7a
VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0xb1
vfs_cache_lookup() at vfs_cache_lookup+0xda
VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xba
lookup() at lookup+0x45e
namei() at namei+0x4cb
kern_statat_vnhook() at kern_statat_vnhook+0x82
kern_statat() at kern_statat+0x15
stat() at stat+0x22
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (188, FreeBSD ELF64, stat), rip = 0x800ec00cc, rsp = 0x7fffffffbec8, rbp = 0 ---

Tracing command sendmail pid 3495 tid 100226 td 0xffffff004fbd2ab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_cv_wait() at _cv_wait+0x15d
zio_wait() at zio_wait+0x61
dbuf_read() at dbuf_read+0x3d6
dmu_buf_hold() at dmu_buf_hold+0xb7
zap_lockdir() at zap_lockdir+0x48
zap_cursor_retrieve() at zap_cursor_retrieve+0x190
zfs_freebsd_readdir() at zfs_freebsd_readdir+0x289
VOP_READDIR_APV() at VOP_READDIR_APV+0xb5
kern_getdirentries() at kern_getdirentries+0x122
getdirentries() at getdirentries+0x23
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (196, FreeBSD ELF64, getdirentries), rip = 0x800e52e2c, rsp = 0x7fffffffc2b8, rbp = 0x1 ---

Tracing command sendmail pid 3494 tid 100224 td 0xffffff004fbac390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_cv_wait() at _cv_wait+0x15d
zio_wait() at zio_wait+0x61
dbuf_read() at dbuf_read+0x3d6
dmu_buf_hold() at dmu_buf_hold+0xb7
zap_lockdir() at zap_lockdir+0x48
zap_lookup_norm() at zap_lookup_norm+0x3a
zap_lookup() at zap_lookup+0x2e
zfs_dirent_lock() at zfs_dirent_lock+0x4f8
zfs_dirlook() at zfs_dirlook+0x5a
zfs_lookup() at zfs_lookup+0x1de
zfs_freebsd_lookup() at zfs_freebsd_lookup+0x7a
VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0xb1
vfs_cache_lookup() at vfs_cache_lookup+0xda
VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xba
lookup() at lookup+0x45e
namei() at namei+0x4cb
kern_statat_vnhook() at kern_statat_vnhook+0x82
kern_statat() at kern_statat+0x15
stat() at stat+0x22
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (188, FreeBSD ELF64, stat), rip = 0x800ec00cc, rsp = 0x7fffffffbec8, rbp = 0 ---

Tracing command cat pid 3210 tid 100222 td 0xffffff002c6fbab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_sleep() at _sleep+0x32b
pipe_read() at pipe_read+0x495
dofileread() at dofileread+0x88
kern_readv() at kern_readv+0x52
read() at read+0x4e
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (3, FreeBSD ELF64, read), rip = 0x80073022c, rsp = 0x7fffffffea38, rbp = 0 ---

Tracing command sh pid 3208 tid 100189 td 0xffffff002c6fb390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_sleep() at _sleep+0x32b
kern_wait() at kern_wait+0x3e7
wait4() at wait4+0x33
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (7, FreeBSD ELF64, wait4), rip = 0x8009304fc, rsp = 0x7fffffffdf68, rbp = 0x7fffffffe128 ---

Tracing command find pid 3207 tid 100190 td 0xffffff002c6fb000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_cv_wait() at _cv_wait+0x15d
zio_wait() at zio_wait+0x61
dbuf_read() at dbuf_read+0x3d6
dnode_hold_impl() at dnode_hold_impl+0xd8
dmu_bonus_hold() at dmu_bonus_hold+0x23
zfs_zget() at zfs_zget+0x213
zfs_dirent_lock() at zfs_dirent_lock+0x49b
zfs_dirlook() at zfs_dirlook+0x5a
zfs_lookup() at zfs_lookup+0x1de
zfs_freebsd_lookup() at zfs_freebsd_lookup+0x7a
VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0xb1
vfs_cache_lookup() at vfs_cache_lookup+0xda
VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xba
lookup() at lookup+0x45e
namei() at namei+0x4cb
kern_statat_vnhook() at kern_statat_vnhook+0x82
kern_statat() at kern_statat+0x15
lstat() at lstat+0x22
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (190, FreeBSD ELF64, lstat), rip = 0x8007290ac, rsp = 0x7fffffffe678, rbp = 0x800a6ddd0 ---

Tracing command sh pid 3203 tid 100194 td 0xffffff002c64b000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_sleep() at _sleep+0x32b
kern_wait() at kern_wait+0x3e7
wait4() at wait4+0x33
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (7, FreeBSD ELF64, wait4), rip = 0x8009304fc, rsp = 0x7fffffffe828, rbp = 0 ---

Tracing command mail pid 3202 tid 100162 td 0xffffff0009613390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_sleep() at _sleep+0x32b
pipe_read() at pipe_read+0x495
dofileread() at dofileread+0x88
kern_readv() at kern_readv+0x52
read() at read+0x4e
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (3, FreeBSD ELF64, read), rip = 0x80073f22c, rsp = 0x7fffffffdcf8, rbp = 0x400 ---

Tracing command sh pid 3201 tid 100230 td 0xffffff004fbc3ab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_sleep() at _sleep+0x32b
kern_wait() at kern_wait+0x3e7
wait4() at wait4+0x33
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (7, FreeBSD ELF64, wait4), rip = 0x8009304fc, rsp = 0x7fffffffe3f8, rbp = 0x7fffffffe5b8 ---

Tracing command sh pid 3200 tid 100227 td 0xffffff004fbd2720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_sleep() at _sleep+0x32b
kern_wait() at kern_wait+0x3e7
wait4() at wait4+0x33
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (7, FreeBSD ELF64, wait4), rip = 0x8009304fc, rsp = 0x7fffffffe0f8, rbp = 0x7fffffffe2b8 ---

Tracing command sh pid 3193 tid 100171 td 0xffffff0009745000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_sleep() at _sleep+0x32b
kern_wait() at kern_wait+0x3e7
wait4() at wait4+0x33
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (7, FreeBSD ELF64, wait4), rip = 0x8009304fc, rsp = 0x7fffffffe8c8, rbp = 0 ---

Tracing command sh pid 3192 tid 100199 td 0xffffff002c6f9ab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_sleep() at _sleep+0x32b
kern_wait() at kern_wait+0x3e7
wait4() at wait4+0x33
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (7, FreeBSD ELF64, wait4), rip = 0x8009304fc, rsp = 0x7fffffffe6b8, rbp = 0x7fffffffe878 ---

Tracing command mail pid 2850 tid 100119 td 0xffffff00093c6000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_sleep() at _sleep+0x32b
pipe_read() at pipe_read+0x495
dofileread() at dofileread+0x88
kern_readv() at kern_readv+0x52
read() at read+0x4e
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (3, FreeBSD ELF64, read), rip = 0x80073f22c, rsp = 0x7fffffffdd18, rbp = 0x400 ---

Tracing command sh pid 2849 tid 100173 td 0xffffff0009753720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_sleep() at _sleep+0x32b
kern_wait() at kern_wait+0x3e7
wait4() at wait4+0x33
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (7, FreeBSD ELF64, wait4), rip = 0x8009304fc, rsp = 0x7fffffffe438, rbp = 0x7fffffffe5f8 ---

Tracing command sh pid 2848 tid 100129 td 0xffffff00093c5390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_sleep() at _sleep+0x32b
kern_wait() at kern_wait+0x3e7
wait4() at wait4+0x33
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (7, FreeBSD ELF64, wait4), rip = 0x8009304fc, rsp = 0x7fffffffe138, rbp = 0x7fffffffe2f8 ---

Tracing command sh pid 2841 tid 100154 td 0xffffff0009616390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_sleep() at _sleep+0x32b
kern_wait() at kern_wait+0x3e7
wait4() at wait4+0x33
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (7, FreeBSD ELF64, wait4), rip = 0x8009304fc, rsp = 0x7fffffffe908, rbp = 0 ---

Tracing command csh pid 2804 tid 100157 td 0xffffff0009615720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_sleep() at _sleep+0x32b
kern_sigsuspend() at kern_sigsuspend+0x93
sigsuspend() at sigsuspend+0x34
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (4, FreeBSD ELF64, write), rip = 0x8009475fc, rsp = 0x7fffffffe628, rbp = 0x800c5d300 ---

Tracing command getty pid 2803 tid 100153 td 0xffffff0009616720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_cv_wait_sig() at _cv_wait_sig+0x169
tty_wait() at tty_wait+0x41
ttydisc_read() at ttydisc_read+0x225
ttydev_read() at ttydev_read+0x89
devfs_read_f() at devfs_read_f+0x7f
dofileread() at dofileread+0x88
kern_readv() at kern_readv+0x52
read() at read+0x4e
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (3, FreeBSD ELF64, read), rip = 0x80084222c, rsp = 0x7fffffffecc8, rbp = 0 ---

Tracing command getty pid 2802 tid 100108 td 0xffffff00093ca390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_cv_wait_sig() at _cv_wait_sig+0x169
tty_wait() at tty_wait+0x41
ttydisc_read() at ttydisc_read+0x225
ttydev_read() at ttydev_read+0x89
devfs_read_f() at devfs_read_f+0x7f
dofileread() at dofileread+0x88
kern_readv() at kern_readv+0x52
read() at read+0x4e
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (3, FreeBSD ELF64, read), rip = 0x80084222c, rsp = 0x7fffffffecc8, rbp = 0 ---

Tracing command getty pid 2801 tid 100164 td 0xffffff00095a5ab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_cv_wait_sig() at _cv_wait_sig+0x169
tty_wait() at tty_wait+0x41
ttydisc_read() at ttydisc_read+0x225
ttydev_read() at ttydev_read+0x89
devfs_read_f() at devfs_read_f+0x7f
dofileread() at dofileread+0x88
kern_readv() at kern_readv+0x52
read() at read+0x4e
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (3, FreeBSD ELF64, read), rip = 0x80084222c, rsp = 0x7fffffffecc8, rbp = 0 ---

Tracing command getty pid 2800 tid 100201 td 0xffffff002c6f9390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_cv_wait_sig() at _cv_wait_sig+0x169
tty_wait() at tty_wait+0x41
ttydisc_read() at ttydisc_read+0x225
ttydev_read() at ttydev_read+0x89
devfs_read_f() at devfs_read_f+0x7f
dofileread() at dofileread+0x88
kern_readv() at kern_readv+0x52
read() at read+0x4e
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (3, FreeBSD ELF64, read), rip = 0x80084222c, rsp = 0x7fffffffecc8, rbp = 0 ---

Tracing command getty pid 2799 tid 100203 td 0xffffff0009747ab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_cv_wait_sig() at _cv_wait_sig+0x169
tty_wait() at tty_wait+0x41
ttydisc_read() at ttydisc_read+0x225
ttydev_read() at ttydev_read+0x89
devfs_read_f() at devfs_read_f+0x7f
dofileread() at dofileread+0x88
kern_readv() at kern_readv+0x52
read() at read+0x4e
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (3, FreeBSD ELF64, read), rip = 0x80084222c, rsp = 0x7fffffffecc8, rbp = 0 ---

Tracing command getty pid 2798 tid 100193 td 0xffffff002c64b390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_cv_wait_sig() at _cv_wait_sig+0x169
tty_wait() at tty_wait+0x41
ttydisc_read() at ttydisc_read+0x225
ttydev_read() at ttydev_read+0x89
devfs_read_f() at devfs_read_f+0x7f
dofileread() at dofileread+0x88
kern_readv() at kern_readv+0x52
read() at read+0x4e
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (3, FreeBSD ELF64, read), rip = 0x80084222c, rsp = 0x7fffffffecc8, rbp = 0 ---

Tracing command getty pid 2797 tid 100120 td 0xffffff0006a75ab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_cv_wait_sig() at _cv_wait_sig+0x169
tty_wait() at tty_wait+0x41
ttydisc_read() at ttydisc_read+0x225
ttydev_read() at ttydev_read+0x89
devfs_read_f() at devfs_read_f+0x7f
dofileread() at dofileread+0x88
kern_readv() at kern_readv+0x52
read() at read+0x4e
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (3, FreeBSD ELF64, read), rip = 0x80084222c, rsp = 0x7fffffffecc8, rbp = 0 ---

Tracing command login pid 2796 tid 100186 td 0xffffff000974e000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_cv_wait() at _cv_wait+0x15d
zio_wait() at zio_wait+0x61
dbuf_read() at dbuf_read+0x3d6
dmu_buf_hold() at dmu_buf_hold+0xb7
zap_lockdir() at zap_lockdir+0x48
zap_lookup_norm() at zap_lookup_norm+0x3a
zap_lookup() at zap_lookup+0x2e
zfs_dirent_lock() at zfs_dirent_lock+0x4f8
zfs_dirlook() at zfs_dirlook+0x5a
zfs_lookup() at zfs_lookup+0x1de
zfs_freebsd_lookup() at zfs_freebsd_lookup+0x7a
VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0xb1
vfs_cache_lookup() at vfs_cache_lookup+0xda
VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xba
lookup() at lookup+0x45e
namei() at namei+0x4cb
vn_open_cred() at vn_open_cred+0x1d7
kern_openat() at kern_openat+0x163
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (5, FreeBSD ELF64, open), rip = 0x80050fe1c, rsp = 0x7fffffffe348, rbp = 0x4010e1 ---

Tracing command login pid 2795 tid 100197 td 0xffffff002c69b390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_sleep() at _sleep+0x32b
kern_wait() at kern_wait+0x3e7
wait4() at wait4+0x33
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (7, FreeBSD ELF64, wait4), rip = 0x8009e64fc, rsp = 0x7fffffffec48, rbp = 0 ---

Tracing command inetd pid 2765 tid 100109 td 0xffffff00093ca000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_cv_wait_sig() at _cv_wait_sig+0x169
seltdwait() at seltdwait+0xa5
kern_select() at kern_select+0x641
select() at select+0x54
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (93, FreeBSD ELF64, select), rip = 0x800a571ac, rsp = 0x7fffffffddc8, rbp = 0x7878787878787878 ---

Tracing command nfscbd pid 2746 tid 100216 td 0xffffff0079d09390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12
_cv_timedwait_sig() at _cv_timedwait_sig+0x177
svc_run_internal() at svc_run_internal+0x803
svc_thread_start() at svc_thread_start+0xb
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0xc, rip = 0x80069b1dc, rsp = 0x7fffffffe9c8, rbp = 0x7fffffffeb60 ---

Tracing command nfscbd pid 2746 tid 100215 td 0xffffff00093ca720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12
_cv_timedwait_sig() at _cv_timedwait_sig+0x177
svc_run_internal() at svc_run_internal+0x803
svc_thread_start() at svc_thread_start+0xb
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0xc, rip = 0x80069b1dc, rsp = 0x7fffffffe9c8, rbp = 0x7fffffffeb60 ---

Tracing command nfscbd pid 2746 tid 100214 td 0xffffff00093caab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12
_cv_timedwait_sig() at _cv_timedwait_sig+0x177
svc_run_internal() at svc_run_internal+0x803
svc_thread_start() at svc_thread_start+0xb
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0xc, rip = 0x80069b1dc, rsp = 0x7fffffffe9c8, rbp = 0x7fffffffeb60 ---

Tracing command nfscbd pid 2746 tid 100195 td 0xffffff002c69bab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12
_cv_timedwait_sig() at _cv_timedwait_sig+0x177
svc_run_internal() at svc_run_internal+0x803
svc_run() at svc_run+0x94
nfscbd_nfsd() at nfscbd_nfsd+0xe8
nfssvc_nfscl() at nfssvc_nfscl+0x60
nfssvc() at nfssvc+0x75
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (155, FreeBSD ELF64, nfssvc), rip = 0x80069b1dc, rsp = 0x7fffffffe9c8, rbp = 0x7fffffffeb60 ---

Tracing command nfscbd pid 2745 tid 100206 td 0xffffff002c67d000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_sleep() at _sleep+0x32b
kern_accept() at kern_accept+0x1a7
accept() at accept+0x61
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (30, FreeBSD ELF64, accept), rip = 0x8006d08ac, rsp = 0x7fffffffe9c8, rbp = 0 ---

Tracing command gdomap pid 2670 tid 100152 td 0xffffff0009616ab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12
_cv_timedwait_sig() at _cv_timedwait_sig+0x177
seltdwait() at seltdwait+0x4f
kern_select() at kern_select+0x641
select() at select+0x54
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (93, FreeBSD ELF64, select), rip = 0x80164c1ac, rsp = 0x7fffffffd018, rbp = 0x400 ---

Tracing command cron pid 2560 tid 100196 td 0xffffff002c69b720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_cv_wait() at _cv_wait+0x15d
zio_wait() at zio_wait+0x61
dbuf_read() at dbuf_read+0x3d6
dmu_buf_hold() at dmu_buf_hold+0xb7
zap_lockdir() at zap_lockdir+0x48
zap_lookup_norm() at zap_lookup_norm+0x3a
zap_lookup() at zap_lookup+0x2e
zfs_dirent_lock() at zfs_dirent_lock+0x4f8
zfs_dirlook() at zfs_dirlook+0x5a
zfs_lookup() at zfs_lookup+0x1de
zfs_freebsd_lookup() at zfs_freebsd_lookup+0x7a
VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0xb1
vfs_cache_lookup() at vfs_cache_lookup+0xda
VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xba
lookup() at lookup+0x45e
namei() at namei+0x4cb
kern_statat_vnhook() at kern_statat_vnhook+0x82
kern_statat() at kern_statat+0x15
stat() at stat+0x22
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (188, FreeBSD ELF64, stat), rip = 0x80093f0cc, rsp = 0x7fffffffe7e8, rbp = 0 ---

Tracing command sendmail pid 2553 tid 100205 td 0xffffff002c67d390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_sleep() at _sleep+0x32b
kern_sigsuspend() at kern_sigsuspend+0x93
sigsuspend() at sigsuspend+0x34
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (4, FreeBSD ELF64, write), rip = 0x800e395fc, rsp = 0x7fffffffcd88, rbp = 0x1 ---

Tracing command sendmail pid 2547 tid 100204 td 0xffffff0009747720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12
_cv_timedwait_sig() at _cv_timedwait_sig+0x177
seltdwait() at seltdwait+0x4f
kern_select() at kern_select+0x641
select() at select+0x54
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (93, FreeBSD ELF64, select), rip = 0x800ece1ac, rsp = 0x7fffffffc208, rbp = 0x7fffffffc2a0 ---

Tracing command cupsd pid 2499 tid 100125 td 0xffffff00093c5720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_sleep() at _sleep+0x32b
kern_kevent() at kern_kevent+0x333
kevent() at kevent+0x85
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (363, FreeBSD ELF64, kevent), rip = 0x8011a8dec, rsp = 0x7fffffffe7e8, rbp = 0x1 ---

Tracing command pcscd pid 2188 tid 100213 td 0xffffff002c73c390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_sleep() at _sleep+0x32b
pipe_read() at pipe_read+0x495
dofileread() at dofileread+0x88
kern_readv() at kern_readv+0x52
read() at read+0x4e
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (3, FreeBSD ELF64, read), rip = 0x80095f22c, rsp = 0x7fffff9fdf68, rbp = 0x800c0ac88 ---

Tracing command pcscd pid 2188 tid 100211 td 0xffffff002c73cab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12
_sleep() at _sleep+0x33e
kern_nanosleep() at kern_nanosleep+0x10b
nanosleep() at nanosleep+0x61
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x800945f5c, rsp = 0x7fffffbfef28, rbp = 0 ---

Tracing command pcscd pid 2188 tid 100122 td 0xffffff00093c5ab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12
_cv_timedwait_sig() at _cv_timedwait_sig+0x177
seltdwait() at seltdwait+0x4f
kern_select() at kern_select+0x641
select() at select+0x54
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (93, FreeBSD ELF64, select), rip = 0x80095f1ac, rsp = 0x7fffffffeac8, rbp = 0x7 ---

Tracing command dkim-filter pid 2170 tid 100210 td 0xffffff002c738000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_sleep() at _sleep+0x32b
kern_sigtimedwait() at kern_sigtimedwait+0x4dd
sigwait() at sigwait+0x63
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (429, FreeBSD ELF64, sigwait), rip = 0x800dc5f1c, rsp = 0x7fffff7fcf38, rbp = 0x80100ac80 ---

Tracing command dkim-filter pid 2170 tid 100209 td 0xffffff002c738390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_sleep() at _sleep+0x32b
kern_sigtimedwait() at kern_sigtimedwait+0x4dd
sigwait() at sigwait+0x63
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (429, FreeBSD ELF64, sigwait), rip = 0x800dc5f1c, rsp = 0x7fffff9fdf48, rbp = 0x80100ae40 ---

Tracing command dkim-filter pid 2170 tid 100208 td 0xffffff002c738720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_cv_wait_sig() at _cv_wait_sig+0x169
seltdwait() at seltdwait+0xa5
kern_select() at kern_select+0x641
select() at select+0x54
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (93, FreeBSD ELF64, select), rip = 0x800e5b1ac, rsp = 0x7fffffbfeca8, rbp = 0x8 ---

Tracing command dkim-filter pid 2170 tid 100169 td 0xffffff0009745720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12
_cv_timedwait_sig() at _cv_timedwait_sig+0x177
seltdwait() at seltdwait+0x4f
poll() at poll+0x3f4
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (209, FreeBSD ELF64, poll), rip = 0x800e0ad3c, rsp = 0x7ffffffee148, rbp = 0 ---

Tracing command racoon pid 2091 tid 100188 td 0xffffff0009618720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12
_cv_timedwait_sig() at _cv_timedwait_sig+0x177
seltdwait() at seltdwait+0x4f
kern_select() at kern_select+0x641
select() at select+0x54
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (93, FreeBSD ELF64, select), rip = 0x800eac1ac, rsp = 0x7fffffffe7d8, rbp = 0x7fffffffecb0 ---

Tracing command privoxy pid 2083 tid 100128 td 0xffffff0009503000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_sleep() at _sleep+0x32b
kern_accept() at kern_accept+0x1a7
accept() at accept+0x61
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (30, FreeBSD ELF64, accept), rip = 0x800b6f8ac, rsp = 0x7fffffffd658, rbp = 0x7fffffffd6d0 ---

Tracing command ifdhandler pid 2071 tid 100168 td 0xffffff0009745ab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12
_cv_timedwait_sig() at _cv_timedwait_sig+0x177
seltdwait() at seltdwait+0x4f
poll() at poll+0x3f4
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (209, FreeBSD ELF64, poll), rip = 0x80091cd3c, rsp = 0x7fffffffd378, rbp = 0x7fffffffe3c0 ---

Tracing command rarpd pid 2003 tid 100185 td 0xffffff000974e390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_cv_wait_sig() at _cv_wait_sig+0x169
seltdwait() at seltdwait+0xa5
kern_select() at kern_select+0x641
select() at select+0x54
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (93, FreeBSD ELF64, select), rip = 0x8007311ac, rsp = 0x7fffffffea88, rbp = 0 ---

Tracing command ntpd pid 1991 tid 100191 td 0xffffff002c64bab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_cv_wait() at _cv_wait+0x15d
zio_wait() at zio_wait+0x61
dbuf_read() at dbuf_read+0x3d6
dmu_buf_hold() at dmu_buf_hold+0xb7
zap_lockdir() at zap_lockdir+0x48
zap_lookup_norm() at zap_lookup_norm+0x3a
zap_lookup() at zap_lookup+0x2e
zfs_dirent_lock() at zfs_dirent_lock+0x4f8
zfs_dirlook() at zfs_dirlook+0x5a
zfs_lookup() at zfs_lookup+0x1de
zfs_freebsd_lookup() at zfs_freebsd_lookup+0x7a
VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0xb1
vfs_cache_lookup() at vfs_cache_lookup+0xda
VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xba
lookup() at lookup+0x45e
namei() at namei+0x4cb
vn_open_cred() at vn_open_cred+0xc2
kern_openat() at kern_openat+0x163
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (5, FreeBSD ELF64, open), rip = 0x800d2c12c, rsp = 0x7fffffffeaa8, rbp = 0x8 ---

Tracing command hcsecd pid 1946 tid 100181 td 0xffffff0009753390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_sleep() at _sleep+0x32b
soreceive_generic() at soreceive_generic+0x1005
kern_recvit() at kern_recvit+0x17b
recvit() at recvit+0x17
recvfrom() at recvfrom+0x7b
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (29, FreeBSD ELF64, recvfrom), rip = 0x8007f5abc, rsp = 0x7fffffffea38, rbp = 0x3 ---

Tracing command bootparamd pid 1939 tid 100110 td 0xffffff00093c8ab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12
_cv_timedwait_sig() at _cv_timedwait_sig+0x177
seltdwait() at seltdwait+0x4f
kern_select() at kern_select+0x641
select() at select+0x54
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (93, FreeBSD ELF64, select), rip = 0x8007301ac, rsp = 0x7fffffffeaf8, rbp = 0x800879a80 ---

Tracing command tor pid 1911 tid 100121 td 0xffffff000950a390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12
_sleep() at _sleep+0x33e
kern_kevent() at kern_kevent+0x333
kevent() at kevent+0x85
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (363, FreeBSD ELF64, kevent), rip = 0x800ee6dec, rsp = 0x7fffffffe9d8, rbp = 0x801241000 ---

Tracing command rpc.statd pid 1837 tid 100118 td 0xffffff00093c6390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12
_cv_timedwait_sig() at _cv_timedwait_sig+0x177
seltdwait() at seltdwait+0x4f
kern_select() at kern_select+0x641
select() at select+0x54
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (93, FreeBSD ELF64, select), rip = 0x80083a1ac, rsp = 0x7fffffffeb58, rbp = 0x800983a80 ---

Tracing command nfsd pid 1830 tid 100180 td 0xffffff0009510000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12
_cv_timedwait_sig() at _cv_timedwait_sig+0x177
svc_run_internal() at svc_run_internal+0x803
svc_thread_start() at svc_thread_start+0xb
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0xc, rip = 0x80069e1dc, rsp = 0x7fffffffe6d8, rbp = 0x5 ---

Tracing command nfsd pid 1830 tid 100179 td 0xffffff0009510390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12
_cv_timedwait_sig() at _cv_timedwait_sig+0x177
svc_run_internal() at svc_run_internal+0x803
svc_thread_start() at svc_thread_start+0xb
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0xc, rip = 0x80069e1dc, rsp = 0x7fffffffe6d8, rbp = 0x5 ---

Tracing command nfsd pid 1830 tid 100178 td 0xffffff0009510720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12
_cv_timedwait_sig() at _cv_timedwait_sig+0x177
svc_run_internal() at svc_run_internal+0x803
svc_thread_start() at svc_thread_start+0xb
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0xc, rip = 0x80069e1dc, rsp = 0x7fffffffe6d8, rbp = 0x5 ---

Tracing command nfsd pid 1830 tid 100117 td 0xffffff0006a79000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12
_cv_timedwait_sig() at _cv_timedwait_sig+0x177
svc_run_internal() at svc_run_internal+0x803
svc_run() at svc_run+0x94
nfssvc_nfsd() at nfssvc_nfsd+0x139
nfssvc_nfsserver() at nfssvc_nfsserver+0x53
nfssvc() at nfssvc+0x75
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (155, FreeBSD ELF64, nfssvc), rip = 0x80069e1dc, rsp = 0x7fffffffe6d8, rbp = 0x5 ---

Tracing command nfsd pid 1829 tid 100155 td 0xffffff0009616000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_cv_wait_sig() at _cv_wait_sig+0x169
seltdwait() at seltdwait+0xa5
kern_select() at kern_select+0x641
select() at select+0x54
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (93, FreeBSD ELF64, select), rip = 0x8007321ac, rsp = 0x7fffffffe978, rbp = 0x7fffffffec50 ---

Tracing command mountd pid 1827 tid 100130 td 0xffffff0009511ab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_cv_wait_sig() at _cv_wait_sig+0x169
seltdwait() at seltdwait+0xa5
kern_select() at kern_select+0x641
select() at select+0x54
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (93, FreeBSD ELF64, select), rip = 0x8008451ac, rsp = 0x7fffffffec48, rbp = 0x7fffffffed40 ---

Tracing command gssd pid 1794 tid 100116 td 0xffffff00093c6720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12
_cv_timedwait_sig() at _cv_timedwait_sig+0x177
seltdwait() at seltdwait+0x4f
kern_select() at kern_select+0x641
select() at select+0x54
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (93, FreeBSD ELF64, select), rip = 0x80083b1ac, rsp = 0x7fffffffea28, rbp = 0x800984a80 ---

Tracing command nfsuserd pid 1788 tid 100127 td 0xffffff0009503390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12
_cv_timedwait_sig() at _cv_timedwait_sig+0x177
seltdwait() at seltdwait+0x4f
kern_select() at kern_select+0x641
select() at select+0x54
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (93, FreeBSD ELF64, select), rip = 0x8007301ac, rsp = 0x7fffffffe9f8, rbp = 0x800879a80 ---

Tracing command nfsuserd pid 1787 tid 100163 td 0xffffff0009613000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12
_cv_timedwait_sig() at _cv_timedwait_sig+0x177
seltdwait() at seltdwait+0x4f
kern_select() at kern_select+0x641
select() at select+0x54
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (93, FreeBSD ELF64, select), rip = 0x8007301ac, rsp = 0x7fffffffe9f8, rbp = 0x800879a80 ---

Tracing command nfsuserd pid 1786 tid 100170 td 0xffffff0009745390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12
_cv_timedwait_sig() at _cv_timedwait_sig+0x177
seltdwait() at seltdwait+0x4f
kern_select() at kern_select+0x641
select() at select+0x54
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (93, FreeBSD ELF64, select), rip = 0x8007301ac, rsp = 0x7fffffffe9f8, rbp = 0x800879a80 ---

Tracing command nfsuserd pid 1785 tid 100111 td 0xffffff00093c8720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12
_cv_timedwait_sig() at _cv_timedwait_sig+0x177
seltdwait() at seltdwait+0x4f
kern_select() at kern_select+0x641
select() at select+0x54
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (93, FreeBSD ELF64, select), rip = 0x8007301ac, rsp = 0x7fffffffe9f8, rbp = 0x800879a80 ---

Tracing command nfsuserd pid 1784 tid 100161 td 0xffffff0009613720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_sleep() at _sleep+0x32b
kern_sigsuspend() at kern_sigsuspend+0x93
sigsuspend() at sigsuspend+0x34
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (4, FreeBSD ELF64, write), rip = 0x80069b5fc, rsp = 0x7fffffffeb68, rbp = 0x4 ---

Tracing command amd pid 1731 tid 100159 td 0xffffff0009615000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_cv_wait() at _cv_wait+0x15d
dbuf_read() at dbuf_read+0x34e
dmu_buf_hold() at dmu_buf_hold+0xb7
zap_lockdir() at zap_lockdir+0x48
zap_lookup_norm() at zap_lookup_norm+0x3a
zap_lookup() at zap_lookup+0x2e
zfs_dirent_lock() at zfs_dirent_lock+0x4f8
zfs_dirlook() at zfs_dirlook+0x5a
zfs_lookup() at zfs_lookup+0x1de
zfs_freebsd_lookup() at zfs_freebsd_lookup+0x7a
VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0xb1
vfs_cache_lookup() at vfs_cache_lookup+0xda
VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xba
lookup() at lookup+0x45e
namei() at namei+0x4cb
vn_open_cred() at vn_open_cred+0x1d7
kern_openat() at kern_openat+0x163
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (5, FreeBSD ELF64, open), rip = 0x80084c12c, rsp = 0x7fffffffe848, rbp = 0x4 ---

Tracing command rpcbind pid 1700 tid 100112 td 0xffffff00093c8390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12
_cv_timedwait_sig() at _cv_timedwait_sig+0x177
seltdwait() at seltdwait+0x4f
poll() at poll+0x3f4
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (209, FreeBSD ELF64, poll), rip = 0x8008fed3c, rsp = 0x7fffffffcad8, rbp = 0x800c120c0 ---

Tracing command named pid 1683 tid 100177 td 0xffffff0009510ab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_sleep() at _sleep+0x32b
kern_kevent() at kern_kevent+0x333
kevent() at kevent+0x85
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (363, FreeBSD ELF64, kevent), rip = 0x800c5adec, rsp = 0x7fffff5fbe08, rbp = 0x14 ---

Tracing command named pid 1683 tid 100176 td 0xffffff0009511000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12
_sleep() at _sleep+0x33e
do_cv_wait() at do_cv_wait+0x60c
__umtx_op_cv_wait() at __umtx_op_cv_wait+0x51
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800a83adc, rsp = 0x7fffff7fcdb8, rbp = 0x800e0ac80 ---

Tracing command named pid 1683 tid 100175 td 0xffffff0009511390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_sleep() at _sleep+0x32b
do_cv_wait() at do_cv_wait+0x7fd
__umtx_op_cv_wait() at __umtx_op_cv_wait+0x51
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800a83adc, rsp = 0x7fffff9fdeb8, rbp = 0x800e0ae40 ---

Tracing command named pid 1683 tid 100174 td 0xffffff0009511720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_sleep() at _sleep+0x32b
do_cv_wait() at do_cv_wait+0x7fd
__umtx_op_cv_wait() at __umtx_op_cv_wait+0x51
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800a83adc, rsp = 0x7fffffbfeeb8, rbp = 0x800e9d1c0 ---

Tracing command named pid 1683 tid 100160 td 0xffffff0009613ab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_sleep() at _sleep+0x32b
kern_sigtimedwait() at kern_sigtimedwait+0x4dd
sigwait() at sigwait+0x63
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (429, FreeBSD ELF64, sigwait), rip = 0x800bddf1c, rsp = 0x7fffffffeb88, rbp = 0x800e041c0 ---

Tracing command syslogd pid 1590 tid 100151 td 0xffffff0009618000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_cv_wait() at _cv_wait+0x15d
zio_wait() at zio_wait+0x61
dbuf_read() at dbuf_read+0x3d6
dmu_tx_check_ioerr() at dmu_tx_check_ioerr+0x84
dmu_tx_count_write() at dmu_tx_count_write+0x4c
dmu_tx_hold_write() at dmu_tx_hold_write+0x43
zfs_freebsd_write() at zfs_freebsd_write+0x3e6
VOP_WRITE_APV() at VOP_WRITE_APV+0x129
vn_write() at vn_write+0x281
dofilewrite() at dofilewrite+0x7a
kern_writev() at kern_writev+0x52
writev() at writev+0x3a
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (121, FreeBSD ELF64, writev), rip = 0x80082f39c, rsp = 0x7fffffffd398, rbp = 0 ---

Tracing command dhclient pid 1363 tid 100115 td 0xffffff00093c6ab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_timedwait_sig() at sleepq_timedwait_sig+0x12
_cv_timedwait_sig() at _cv_timedwait_sig+0x177
seltdwait() at seltdwait+0x4f
poll() at poll+0x3f4
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (209, FreeBSD ELF64, poll), rip = 0x8006eed3c, rsp = 0x7fffffffece8, rbp = 0x2 ---

Tracing command dhclient pid 1260 tid 100165 td 0xffffff00095a5720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_cv_wait_sig() at _cv_wait_sig+0x169
seltdwait() at seltdwait+0xa5
poll() at poll+0x3f4
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (209, FreeBSD ELF64, poll), rip = 0x8006eed3c, rsp = 0x7fffffffece8, rbp = 0x5 ---

Tracing command devd pid 1209 tid 100126 td 0xffffff0009503720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_cv_wait_sig() at _cv_wait_sig+0x169
seltdwait() at seltdwait+0xa5
kern_select() at kern_select+0x641
select() at select+0x54
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (93, FreeBSD ELF64, select), rip = 0x43da6c, rsp = 0x7fffffffe898, rbp = 0x7fffffffe8b0 ---

Tracing command wpa_supplicant pid 545 tid 100124 td 0xffffff0009503ab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_cv_wait_sig() at _cv_wait_sig+0x169
seltdwait() at seltdwait+0xa5
kern_select() at kern_select+0x641
select() at select+0x54
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (93, FreeBSD ELF64, select), rip = 0x800c7d1ac, rsp = 0x7fffffffeb38, rbp = 0x800e07080 ---

Tracing command adjkerntz pid 133 tid 100114 td 0xffffff0006a79390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_sleep() at _sleep+0x32b
kern_sigsuspend() at kern_sigsuspend+0x93
sigsuspend() at sigsuspend+0x34
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (4, FreeBSD ELF64, write), rip = 0x80069a5fc, rsp = 0x7fffffffec58, rbp = 0x7fffffffecd0 ---

Tracing command flowcleaner pid 26 tid 100076 td 0xffffff00069d2390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_timedwait() at sleepq_timedwait+0x46
_cv_timedwait() at _cv_timedwait+0x177
flowtable_cleaner() at flowtable_cleaner+0xdc
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803dcb9d30, rbp = 0 ---

Tracing command softdepflush pid 25 tid 100075 td 0xffffff00069d2720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_timedwait() at sleepq_timedwait+0x46
_sleep() at _sleep+0x226
softdep_flush() at softdep_flush+0x253
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803dcb4d30, rbp = 0 ---

Tracing command vnlru pid 24 tid 100074 td 0xffffff00069d2ab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_timedwait() at sleepq_timedwait+0x46
_sleep() at _sleep+0x226
vnlru_proc() at vnlru_proc+0x62e
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803dcafd30, rbp = 0 ---

Tracing command syncer pid 23 tid 100073 td 0xffffff00069d3000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_cv_wait() at _cv_wait+0x15d
zio_wait() at zio_wait+0x61
zil_commit() at zil_commit+0x3de
zfs_sync() at zfs_sync+0xa5
sync_fsync() at sync_fsync+0x134
VOP_FSYNC_APV() at VOP_FSYNC_APV+0xb5
sync_vnode() at sync_vnode+0x143
sched_sync() at sched_sync+0x1c6
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803dcaad30, rbp = 0 ---

Tracing command bufdaemon pid 22 tid 100072 td 0xffffff00069d3390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_timedwait() at sleepq_timedwait+0x46
_sleep() at _sleep+0x226
buf_daemon() at buf_daemon+0x16a
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803dca5d30, rbp = 0 ---

Tracing command pagezero pid 21 tid 100071 td 0xffffff00069d3720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_timedwait() at sleepq_timedwait+0x46
_sleep() at _sleep+0x226
vm_pagezero() at vm_pagezero+0x71
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803dca0d30, rbp = 0 ---

Tracing command vmdaemon pid 20 tid 100070 td 0xffffff00069d3ab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
vm_daemon() at vm_daemon+0x4d
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803dc9bd30, rbp = 0 ---

Tracing command pagedaemon pid 19 tid 100069 td 0xffffff00069d6000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_timedwait() at sleepq_timedwait+0x46
_sleep() at _sleep+0x226
vm_pageout() at vm_pageout+0x89f
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803dc96d30, rbp = 0 ---

Tracing command xpt_thrd pid 18 tid 100068 td 0xffffff00037b8000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
xpt_scanner_thread() at xpt_scanner_thread+0xd5
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803dc61d30, rbp = 0 ---

Tracing command zfskern pid 9 tid 100103 td 0xffffff002ad0d390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
vdev_geom_worker() at vdev_geom_worker+0xed
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8040039d30, rbp = 0 ---

Tracing command zfskern pid 9 tid 100106 td 0xffffff00069d1390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_cv_wait() at _cv_wait+0x15d
zio_wait() at zio_wait+0x61
dbuf_read() at dbuf_read+0x3d6
dbuf_findbp() at dbuf_findbp+0xe5
dbuf_hold_impl() at dbuf_hold_impl+0xc2
dbuf_hold() at dbuf_hold+0x1b
dmu_buf_hold() at dmu_buf_hold+0x75
bplist_iterate() at bplist_iterate+0xa1
spa_sync() at spa_sync+0x261
txg_sync_thread() at txg_sync_thread+0x285
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8040059d30, rbp = 0 ---

Tracing command zfskern pid 9 tid 100105 td 0xffffff00069d1720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_cv_wait() at _cv_wait+0x15d
txg_thread_wait() at txg_thread_wait+0x7d
txg_quiesce_thread() at txg_quiesce_thread+0xb5
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8040055d30, rbp = 0 ---

Tracing command zfskern pid 9 tid 100066 td 0xffffff00037b8720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_timedwait() at sleepq_timedwait+0x46
_cv_timedwait() at _cv_timedwait+0x177
l2arc_feed_thread() at l2arc_feed_thread+0x143
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803dc57d30, rbp = 0 ---

Tracing command zfskern pid 9 tid 100065 td 0xffffff00037b8ab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_timedwait() at sleepq_timedwait+0x46
_cv_timedwait() at _cv_timedwait+0x177
arc_reclaim_thread() at arc_reclaim_thread+0x2a5
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803dc52d30, rbp = 0 ---

Tracing command acpi_cooling0 pid 17 tid 100062 td 0xffffff00037b9720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_timedwait() at sleepq_timedwait+0x46
_sleep() at _sleep+0x226
acpi_tz_cooling_thread() at acpi_tz_cooling_thread+0x11a
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803dc3fd30, rbp = 0 ---

Tracing command acpi_thermal pid 16 tid 100061 td 0xffffff00037b9ab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_timedwait() at sleepq_timedwait+0x46
_sleep() at _sleep+0x226
acpi_tz_thread() at acpi_tz_thread+0x200
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803dc3ad30, rbp = 0 ---

Tracing command fw0_probe pid 8 tid 100058 td 0xffffff00037bc720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_sleep() at _sleep+0x32b
fw_bus_probe_thread() at fw_bus_probe_thread+0xab
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803bd00d30, rbp = 0 ---

Tracing command cbb0 event thread pid 7 tid 100055 td 0xffffff0003757720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
cbb_event_thread() at cbb_event_thread+0x1d3
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803bceed30, rbp = 0 ---

Tracing command usb pid 15 tid 100054 td 0xffffff0003757ab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_cv_wait() at _cv_wait+0x15d
usb_process() at usb_process+0x15f
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803bce9d30, rbp = 0 ---

Tracing command usb pid 15 tid 100053 td 0xffffff000378b000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_cv_wait() at _cv_wait+0x15d
usb_process() at usb_process+0x15f
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803bce4d30, rbp = 0 ---

Tracing command usb pid 15 tid 100052 td 0xffffff000378b390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_cv_wait() at _cv_wait+0x15d
usb_process() at usb_process+0x15f
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803bcdfd30, rbp = 0 ---

Tracing command usb pid 15 tid 100051 td 0xffffff000378b720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_cv_wait() at _cv_wait+0x15d
usb_process() at usb_process+0x15f
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803bcdad30, rbp = 0 ---

Tracing command usb pid 15 tid 100049 td 0xffffff000378c000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_cv_wait() at _cv_wait+0x15d
usb_process() at usb_process+0x15f
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803bccfd30, rbp = 0 ---

Tracing command usb pid 15 tid 100048 td 0xffffff000378c390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_cv_wait() at _cv_wait+0x15d
usb_process() at usb_process+0x15f
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803bccad30, rbp = 0 ---

Tracing command usb pid 15 tid 100047 td 0xffffff000378c720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_cv_wait() at _cv_wait+0x15d
usb_process() at usb_process+0x15f
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803bcc5d30, rbp = 0 ---

Tracing command usb pid 15 tid 100046 td 0xffffff000378cab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_cv_wait() at _cv_wait+0x15d
usb_process() at usb_process+0x15f
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803bcc0d30, rbp = 0 ---

Tracing command usb pid 15 tid 100045 td 0xffffff000353fab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_cv_wait() at _cv_wait+0x15d
usb_process() at usb_process+0x15f
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803bcbad30, rbp = 0 ---

Tracing command usb pid 15 tid 100044 td 0xffffff0003754000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_cv_wait() at _cv_wait+0x15d
usb_process() at usb_process+0x15f
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803bcb5d30, rbp = 0 ---

Tracing command usb pid 15 tid 100043 td 0xffffff0003754390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_cv_wait() at _cv_wait+0x15d
usb_process() at usb_process+0x15f
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803bcb0d30, rbp = 0 ---

Tracing command usb pid 15 tid 100042 td 0xffffff0003754720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_cv_wait() at _cv_wait+0x15d
usb_process() at usb_process+0x15f
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803bcabd30, rbp = 0 ---

Tracing command usb pid 15 tid 100041 td 0xffffff0003754ab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_cv_wait() at _cv_wait+0x15d
usb_process() at usb_process+0x15f
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803bca5d30, rbp = 0 ---

Tracing command usb pid 15 tid 100040 td 0xffffff0003755000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_cv_wait() at _cv_wait+0x15d
usb_process() at usb_process+0x15f
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803bca0d30, rbp = 0 ---

Tracing command usb pid 15 tid 100039 td 0xffffff0003755390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_cv_wait() at _cv_wait+0x15d
usb_process() at usb_process+0x15f
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803bc9bd30, rbp = 0 ---

Tracing command usb pid 15 tid 100038 td 0xffffff0003755720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_cv_wait() at _cv_wait+0x15d
usb_process() at usb_process+0x15f
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803bc96d30, rbp = 0 ---

Tracing command usb pid 15 tid 100037 td 0xffffff0003755ab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_cv_wait() at _cv_wait+0x15d
usb_process() at usb_process+0x15f
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803bc90d30, rbp = 0 ---

Tracing command usb pid 15 tid 100036 td 0xffffff0003757000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_cv_wait() at _cv_wait+0x15d
usb_process() at usb_process+0x15f
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803bc8bd30, rbp = 0 ---

Tracing command usb pid 15 tid 100035 td 0xffffff000350b720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_cv_wait() at _cv_wait+0x15d
usb_process() at usb_process+0x15f
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803bc86d30, rbp = 0 ---

Tracing command usb pid 15 tid 100034 td 0xffffff000350bab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_cv_wait() at _cv_wait+0x15d
usb_process() at usb_process+0x15f
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803bc81d30, rbp = 0 ---

Tracing command yarrow pid 14 tid 100017 td 0xffffff0003383000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_timedwait() at sleepq_timedwait+0x46
_sleep() at _sleep+0x226
random_kthread() at random_kthread+0x1ba
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8000071d30, rbp = 0 ---

Tracing command crypto returns pid 6 tid 100016 td 0xffffff0003383390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
crypto_ret_proc() at crypto_ret_proc+0x1b3
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff800006cd30, rbp = 0 ---

Tracing command crypto pid 5 tid 100015 td 0xffffff0003383720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
crypto_proc() at crypto_proc+0x1cf
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8000067d30, rbp = 0 ---

Tracing command g_down pid 4 tid 100013 td 0xffffff0003384000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
g_io_schedule_down() at g_io_schedule_down+0x1f4
g_down_procbody() at g_down_procbody+0x6f
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff800005dd30, rbp = 0 ---

Tracing command g_up pid 3 tid 100012 td 0xffffff0003384390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
g_io_schedule_up() at g_io_schedule_up+0x12e
g_up_procbody() at g_up_procbody+0x6f
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8000058d30, rbp = 0 ---

Tracing command g_event pid 2 tid 100011 td 0xffffff000336b390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_timedwait() at sleepq_timedwait+0x46
_sleep() at _sleep+0x226
g_event_procbody() at g_event_procbody+0xc3
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8000053d30, rbp = 0 ---

Tracing command ng_queue pid 13 tid 100010 td 0xffffff000336b720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
ngthread() at ngthread+0x59
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff800004ed30, rbp = 0 ---

Tracing command ng_queue pid 13 tid 100009 td 0xffffff000336bab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
ngthread() at ngthread+0x59
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8000049d30, rbp = 0 ---

Tracing command intr pid 12 tid 100064 td 0xffffff00037b9000
fork_trampoline() at fork_trampoline

Tracing command intr pid 12 tid 100063 td 0xffffff00037b9390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
ithread_loop() at ithread_loop+0x23e
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803dc44d30, rbp = 0 ---

Tracing command intr pid 12 tid 100060 td 0xffffff00037bc000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
ithread_loop() at ithread_loop+0x23e
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803bd3dd30, rbp = 0 ---

Tracing command intr pid 12 tid 100059 td 0xffffff00037bc390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
ithread_loop() at ithread_loop+0x23e
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803bd38d30, rbp = 0 ---

Tracing command intr pid 12 tid 100056 td 0xffffff0003757390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
ithread_loop() at ithread_loop+0x23e
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803bcf3d30, rbp = 0 ---

Tracing command intr pid 12 tid 100050 td 0xffffff000378bab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
ithread_loop() at ithread_loop+0x23e
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803bcd5d30, rbp = 0 ---

Tracing command intr pid 12 tid 100033 td 0xffffff000353d000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
ithread_loop() at ithread_loop+0x23e
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803bc7cd30, rbp = 0 ---

Tracing command intr pid 12 tid 100031 td 0xffffff000353d720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
ithread_loop() at ithread_loop+0x23e
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8000188d30, rbp = 0 ---

Tracing command intr pid 12 tid 100029 td 0xffffff000353f000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
ithread_loop() at ithread_loop+0x23e
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff80000add30, rbp = 0 ---

Tracing command intr pid 12 tid 100028 td 0xffffff000353f390
fork_trampoline() at fork_trampoline

Tracing command intr pid 12 tid 100027 td 0xffffff000353f720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
ithread_loop() at ithread_loop+0x23e
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff80000a3d30, rbp = 0 ---

Tracing command intr pid 12 tid 100023 td 0xffffff0003508390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
ithread_loop() at ithread_loop+0x23e
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff800008fd30, rbp = 0 ---

Tracing command intr pid 12 tid 100022 td 0xffffff0003508720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
ithread_loop() at ithread_loop+0x23e
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff800008ad30, rbp = 0 ---

Tracing command intr pid 12 tid 100020 td 0xffffff000350b000
fork_trampoline() at fork_trampoline

Tracing command intr pid 12 tid 100018 td 0xffffff000337aab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
ithread_loop() at ithread_loop+0x23e
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8000076d30, rbp = 0 ---

Tracing command intr pid 12 tid 100008 td 0xffffff000337a000
fork_trampoline() at fork_trampoline

Tracing command intr pid 12 tid 100007 td 0xffffff000337a390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
ithread_loop() at ithread_loop+0x23e
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff800003fd30, rbp = 0 ---

Tracing command intr pid 12 tid 100006 td 0xffffff000337a720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
ithread_loop() at ithread_loop+0x23e
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff800003ad30, rbp = 0 ---

Tracing command intr pid 12 tid 100005 td 0xffffff000336a000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
ithread_loop() at ithread_loop+0x23e
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8000035d30, rbp = 0 ---

Tracing command idle pid 11 tid 100004 td 0xffffff000336a390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sched_preempt() at sched_preempt+0xbc
ipi_bitmap_handler() at ipi_bitmap_handler+0x6d
Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0x9b
--- interrupt, rip = 0xffffffff8090154e, rsp = 0xffffff8000030b50, rbp = 0xffffff8000030b60 ---
acpi_cpu_c1() at acpi_cpu_c1+0x6
acpi_cpu_idle() at acpi_cpu_idle+0x1f0
sched_idletd() at sched_idletd+0x11f
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8000030d30, rbp = 0 ---

Tracing command idle pid 11 tid 100003 td 0xffffff000336a720
cpustop_handler() at cpustop_handler+0x40
ipi_nmi_handler() at ipi_nmi_handler+0x32
trap() at trap+0x28c
nmi_calltrap() at nmi_calltrap+0x8
--- trap 0x13, rip = 0xffffffff8090154e, rsp = 0xffffff800000bfe0, rbp = 0xffffff800002bb60 ---
acpi_cpu_c1() at acpi_cpu_c1+0x6
acpi_cpu_idle() at acpi_cpu_idle+0x1f0
sched_idletd() at sched_idletd+0x11f
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff800002bd30, rbp = 0 ---

Tracing command init pid 1 tid 100002 td 0xffffff000336aab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_catch_signals() at sleepq_catch_signals+0x307
sleepq_wait_sig() at sleepq_wait_sig+0xc
_sleep() at _sleep+0x32b
kern_wait() at kern_wait+0x3e7
wait4() at wait4+0x33
syscall() at syscall+0x118
Xfast_syscall() at Xfast_syscall+0xe1
--- syscall (7, FreeBSD ELF64, wait4), rip = 0x40c04c, rsp = 0x7fffffffe808, rbp = 0x401baa ---

Tracing command audit pid 10 tid 100001 td 0xffffff000336b000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_cv_wait() at _cv_wait+0x15d
audit_worker() at audit_worker+0x77
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8000021d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100212 td 0xffffff002c73c720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff804026bd30, rbp = 0 ---

Tracing command kernel pid 0 tid 100150 td 0xffffff0009618390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8040135d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100149 td 0xffffff000950a720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8040130d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100148 td 0xffffff000950aab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff804012bd30, rbp = 0 ---

Tracing command kernel pid 0 tid 100147 td 0xffffff00095a2000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8040126d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100146 td 0xffffff00095a2390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8040121d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100145 td 0xffffff00095a2720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff804011cd30, rbp = 0 ---

Tracing command kernel pid 0 tid 100144 td 0xffffff00095a2ab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8040117d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100143 td 0xffffff00095a3000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8040112d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100142 td 0xffffff00095a3390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff804010dd30, rbp = 0 ---

Tracing command kernel pid 0 tid 100141 td 0xffffff00095a3720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8040108d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100140 td 0xffffff00095a3ab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8040103d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100139 td 0xffffff00095a4000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff80400fed30, rbp = 0 ---

Tracing command kernel pid 0 tid 100138 td 0xffffff00095a4390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff80400f9d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100137 td 0xffffff00095a4720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff80400f4d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100136 td 0xffffff00095a4ab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff80400efd30, rbp = 0 ---

Tracing command kernel pid 0 tid 100135 td 0xffffff00095a5000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff80400ead30, rbp = 0 ---

Tracing command kernel pid 0 tid 100134 td 0xffffff00095a5390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff80400e5d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100133 td 0xffffff0006a7c720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff80400e0d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100132 td 0xffffff0006a7cab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff80400dbd30, rbp = 0 ---

Tracing command kernel pid 0 tid 100131 td 0xffffff00093c5000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff80400d6d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100107 td 0xffffff00069d1000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff804005ed30, rbp = 0 ---

Tracing command kernel pid 0 tid 100104 td 0xffffff0006a71000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff804003ed30, rbp = 0 ---

Tracing command kernel pid 0 tid 100102 td 0xffffff0006a71720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8040034d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100101 td 0xffffff0006a74000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff804002fd30, rbp = 0 ---

Tracing command kernel pid 0 tid 100100 td 0xffffff0006a74720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff804002ad30, rbp = 0 ---

Tracing command kernel pid 0 tid 100099 td 0xffffff0006a74ab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8040025d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100098 td 0xffffff0006a71ab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8040020d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100097 td 0xffffff0006a75000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff804000cd30, rbp = 0 ---

Tracing command kernel pid 0 tid 100096 td 0xffffff0006a75390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8040002d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100095 td 0xffffff0006a75720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff804001bd30, rbp = 0 ---

Tracing command kernel pid 0 tid 100094 td 0xffffff00069d2000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8040016d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100093 td 0xffffff00069d1ab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8040011d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100092 td 0xffffff0006a79720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8040007d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100091 td 0xffffff0006a79ab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803fffdd30, rbp = 0 ---

Tracing command kernel pid 0 tid 100090 td 0xffffff0006a7a000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803fff8d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100089 td 0xffffff0006a7a390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803ffdad30, rbp = 0 ---

Tracing command kernel pid 0 tid 100088 td 0xffffff0006a7a720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803fff3d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100087 td 0xffffff0006a7aab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803ffeed30, rbp = 0 ---

Tracing command kernel pid 0 tid 100086 td 0xffffff0006a7b000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803ffe9d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100085 td 0xffffff0006a7c000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803ffdfd30, rbp = 0 ---

Tracing command kernel pid 0 tid 100084 td 0xffffff00069d6390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803ffd5d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100083 td 0xffffff0006a7b390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803ffd0d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100082 td 0xffffff0006a7b720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803ffe4d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100081 td 0xffffff0006a7bab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803ffcbd30, rbp = 0 ---

Tracing command kernel pid 0 tid 100080 td 0xffffff0006a7c390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803ffc6d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100079 td 0xffffff00069d6720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803ffc1d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100078 td 0xffffff00069d6ab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8040043d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100077 td 0xffffff0006a74390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8040048d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100067 td 0xffffff00037b8390
kdb_enter() at kdb_enter+0x3d
panic() at panic+0x176
deadlkres() at deadlkres+0x229
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803dc5cd30, rbp = 0 ---

Tracing command kernel pid 0 tid 100057 td 0xffffff00037bcab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
msleep_spin() at msleep_spin+0x201
taskqueue_thread_loop() at taskqueue_thread_loop+0x62
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803bcfbd30, rbp = 0 ---

Tracing command kernel pid 0 tid 100032 td 0xffffff000353d390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
msleep_spin() at msleep_spin+0x201
taskqueue_thread_loop() at taskqueue_thread_loop+0x62
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff803bc76d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100030 td 0xffffff000353dab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8000183d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100026 td 0xffffff0003384720
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
msleep_spin() at msleep_spin+0x201
taskqueue_thread_loop() at taskqueue_thread_loop+0x62
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff800009ed30, rbp = 0 ---

Tracing command kernel pid 0 tid 100025 td 0xffffff0003384ab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
msleep_spin() at msleep_spin+0x201
taskqueue_thread_loop() at taskqueue_thread_loop+0x62
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8000099d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100024 td 0xffffff0003508000
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
msleep_spin() at msleep_spin+0x201
taskqueue_thread_loop() at taskqueue_thread_loop+0x62
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8000094d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100021 td 0xffffff0003508ab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8000085d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100019 td 0xffffff000350b390
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff800007bd30, rbp = 0 ---

Tracing command kernel pid 0 tid 100014 td 0xffffff0003383ab0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_wait() at sleepq_wait+0x46
_sleep() at _sleep+0x241
taskqueue_thread_loop() at taskqueue_thread_loop+0xb0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff8000062d30, rbp = 0 ---

Tracing command kernel pid 0 tid 100000 td 0xffffffff80d252e0
sched_switch() at sched_switch+0x149
mi_switch() at mi_switch+0x21b
sleepq_switch() at sleepq_switch+0xfa
sleepq_timedwait() at sleepq_timedwait+0x46
_sleep() at _sleep+0x226
scheduler() at scheduler+0x28e
mi_startup() at mi_startup+0x59
btext() at btext+0x2c

----------------
end


More information about the freebsd-current mailing list