USB4BSD release candidate number 3 - request for review

Thomas Sparrevohn Thomas.Sparrevohn at btinternet.com
Fri Nov 7 11:46:13 PST 2008


On Thursday 06 November 2008 23:36:14 Hans Petter Selasky wrote:
> On Thursday 06 November 2008, Alexey Shuvaev wrote:
> > db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
> > _witness_debugger() at _witness_debugger+0x49
> > witness_checkorder() at witness_checkorder+0x7e6
> > _sx_xlock() at _sx_xlock+0x44
> > vm_map_lookup() at vm_map_lookup+0x47
> > vm_fault() at vm_fault+0xfe
> > trap_pfault() at trap_pfault+0x1fa
> > trap() at trap+0x201
> > calltrap() at calltrap+0x8
> > --- trap 0xc, rip = 0xffffffff804b2466, rsp = 0xfffffffeba7f8ac0, rbp =
> > 0xfffffffeba7f8b00 --- bcopy() at bcopy+0x16
> > usb2_bdma_pre_sync() at usb2_bdma_pre_sync+0x35
> > usb2_bdma_work_loop() at usb2_bdma_work_loop+0x29b
> > usb2_command_wrapper() at usb2_command_wrapper+0x76
> > usb2_callback_wrapper() at usb2_callback_wrapper+0xfd
> > usb2_command_wrapper() at usb2_command_wrapper+0x76
> > usb2_callback_proc() at usb2_callback_proc+0x68
> > usb2_process() at usb2_process+0xc0
> 
> Hi Alexey,
> 
> This looks more like a busdma problem to me. "bcopy()" is called from 
> within "bus_dmamap_sync()". Maybe you can figure out if it is the source or 
> the destination address that fails?
> 
> Looks like your system has started bouncing data to the 32-bit address range.
> 

I can reproduce an error like it whenever I use DMA on umass device - The error 
somehow are triggered/created by the multiport patches that was introduced
in the ATA framework back in April - For a long time I believed it was an error 
in the ATA frame work until I discovered that If I disabled the umass device
the error disappered 

I am using ZFS on both devices. If I change to usb2_ata the problem disappers.
I think that is because usb2_ata does not use DMA.

I believe the problem is related to the DMA handling in the umass 

see attached fault - In short the problem has been around for a long time
but it seems to be related to the USB stack DMA handling - before usb2
it showed up as a DMA error in the ATA driver - but that was because
the old umass driver somehow "stole" an active DMA 


w2fzz0vc03.aah-go-on.com dumped core - see /var/crash/vmcore.42

Thu Oct 16 19:11:48 BST 2008

FreeBSD w2fzz0vc03.aah-go-on.com 8.0-CURRENT FreeBSD 8.0-CURRENT #1: Wed Oct 15 21:39:04 BST 2008     sandbox at w2fzz0vc03.aah-go-on.com:/usr/home/sandbox/newusb/obj/usr/home/sandbox/newusb/src/sys/GENERIC_usb2  amd64

panic: mutex Giant not owned at /usr/home/sandbox/newusb/src/sys/kern/tty_ttydisc.c:1129

GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...

Unread portion of the kernel message buffer:
Kernel page fault with the following non-sleepable locks held:
exclusive sleep mutex UMASS lock (UMASS lock) r = 0 (0xffffffff80b67e60) locked @ /usr/home/sandbox/newusb/src/sys/dev/usb2/core/usb2_transfer.c:1795
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
_witness_debugger() at _witness_debugger+0x49
witness_warn() at witness_warn+0x232
trap() at trap+0x38f
calltrap() at calltrap+0x8
--- trap 0xc, rip = 0xffffffff807b2a46, rsp = 0xfffffffeb7267ac0, rbp = 0xfffffffeb7267b00 ---
bcopy() at bcopy+0x16
usb2_bdma_pre_sync() at usb2_bdma_pre_sync+0x59
usb2_bdma_work_loop() at usb2_bdma_work_loop+0x2ac
usb2_command_wrapper() at usb2_command_wrapper+0x76
usb2_callback_wrapper() at usb2_callback_wrapper+0x101
usb2_command_wrapper() at usb2_command_wrapper+0x76
usb2_callback_proc() at usb2_callback_proc+0x68
usb2_process() at usb2_process+0xc0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xfffffffeb7267d40, rbp = 0 ---


Fatal trap 12: page fault while in kernel mode
cpuid = 3; apic id = 03
fault virtual address	= 0xfffffffe69a00000
fault code		= supervisor read data, page not present
instruction pointer	= 0x8:0xffffffff807b2a46
stack pointer	        = 0x10:0xfffffffeb7267ac0
frame pointer	        = 0x10:0xfffffffeb7267b00
code segment		= base 0x0, limit 0xfffff, type 0x1b
			= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags	= interrupt enabled, resume, IOPL = 0
current process		= 139 (USBPROC)
lock order reversal: (Giant after non-sleepable)
 1st 0xffffffff80b67e60 UMASS lock (UMASS lock) @ /usr/home/sandbox/newusb/src/sys/dev/usb2/core/usb2_transfer.c:1795
 2nd 0xffffffff80b6bfe0 Giant (Giant) @ /usr/home/sandbox/newusb/src/sys/dev/kbdmux/kbdmux.c:1044
KDB: stack backtrace:
panic: mutex Giant not owned at /usr/home/sandbox/newusb/src/sys/kern/tty_ttydisc.c:1129
cpuid = 3
KDB: stack backtrace:
Physical memory: 4078 MB
Dumping 1154 MB: 1139 1123 1107 1091 1075 1059 1043 1027 1011 995 979 963 947 931 915 899 883 867 851 835 819 803 787 771 755 739 723 707 691 675 659 643 627 611 595 579 563 547 531 515 499 483 467 451 435 419 403 387 371 355 339 323 307 291 275 259 243 227 211 195 179 163 147 131 115 99 83 67 51 35 19 3

Reading symbols from /boot/kernel/zfs.ko...Reading symbols from /bootdir/boot/kernel/zfs.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/zfs.ko
Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from /bootdir/boot/kernel/opensolaris.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/opensolaris.ko
Reading symbols from /boot/kernel/linux.ko...Reading symbols from /bootdir/boot/kernel/linux.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/linux.ko
Reading symbols from /boot/kernel/daemon_saver.ko...Reading symbols from /bootdir/boot/kernel/daemon_saver.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/daemon_saver.ko
#0  doadump () at pcpu.h:196
196	pcpu.h: No such file or directory.
	in pcpu.h
(kgdb) #0  doadump () at pcpu.h:196
#1  0xffffffff801c475c in db_fncall (dummy1=Variable "dummy1" is not available.
)
    at /usr/home/sandbox/newusb/src/sys/ddb/db_command.c:548
#2  0xffffffff801c4a91 in db_command (last_cmdp=0xffffffff80b384a0, cmd_table=Variable "cmd_table" is not available.

) at /usr/home/sandbox/newusb/src/sys/ddb/db_command.c:445
#3  0xffffffff801c4cd9 in db_command_loop ()
    at /usr/home/sandbox/newusb/src/sys/ddb/db_command.c:498
#4  0xffffffff801c6b37 in db_trap (type=Variable "type" is not available.
)
    at /usr/home/sandbox/newusb/src/sys/ddb/db_main.c:229
#5  0xffffffff8051deb2 in kdb_trap (type=12, code=0, tf=0xfffffffeb7267a10)
    at /usr/home/sandbox/newusb/src/sys/kern/subr_kdb.c:534
#6  0xffffffff807b361d in trap_fatal (frame=0xfffffffeb7267a10, eva=Variable "eva" is not available.
)
    at /usr/home/sandbox/newusb/src/sys/amd64/amd64/trap.c:754
#7  0xffffffff807b4371 in trap (frame=0xfffffffeb7267a10)
    at /usr/home/sandbox/newusb/src/sys/amd64/amd64/trap.c:289
#8  0xffffffff807973de in calltrap ()
    at /usr/home/sandbox/newusb/src/sys/amd64/amd64/exception.S:217
#9  0xffffffff807b2a46 in bcopy ()
    at /usr/home/sandbox/newusb/src/sys/amd64/amd64/support.S:123
#10 0xffffffff80791ea0 in _bus_dmamap_sync (dmat=0xffffff0004028400, map=Variable "map" is not available.
)
    at /usr/home/sandbox/newusb/src/sys/amd64/amd64/busdma_machdep.c:922
#11 0xffffffff80426b42 in usb2_bdma_pre_sync (xfer=Variable "xfer" is not available.
)
    at /usr/home/sandbox/newusb/src/sys/dev/usb2/core/usb2_busdma.c:1360
#12 0xffffffff80426f01 in usb2_bdma_work_loop (pq=0xfffffffe40f84000)
    at /usr/home/sandbox/newusb/src/sys/dev/usb2/core/usb2_busdma.c:1297
#13 0xffffffff80439b26 in usb2_command_wrapper (pq=0xfffffffe40f84000, xfer=Variable "xfer" is not available.
)
    at /usr/home/sandbox/newusb/src/sys/dev/usb2/core/usb2_transfer.c:2515
#14 0xffffffff8043c3ed in usb2_callback_wrapper (pq=Variable "pq" is not available.
)
    at /usr/home/sandbox/newusb/src/sys/dev/usb2/core/usb2_transfer.c:1926
#15 0xffffffff80439b26 in usb2_command_wrapper (pq=0xfffffffe40f84028, xfer=Variable "xfer" is not available.
)
    at /usr/home/sandbox/newusb/src/sys/dev/usb2/core/usb2_transfer.c:2515
#16 0xffffffff80439c70 in usb2_callback_proc (_pm=Variable "_pm" is not available.
)
    at /usr/home/sandbox/newusb/src/sys/dev/usb2/core/usb2_transfer.c:1799
#17 0xffffffff804378b4 in usb2_process (arg=Variable "arg" is not available.
)
    at /usr/home/sandbox/newusb/src/sys/dev/usb2/core/usb2_process.c:139
#18 0xffffffff804d314a in fork_exit (
    callout=0xffffffff804377f4 <usb2_process>, arg=0xfffffffe40f840f0, 
    frame=0xfffffffeb7267c90)
    at /usr/home/sandbox/newusb/src/sys/kern/kern_fork.c:810
#19 0xffffffff807977ee in fork_trampoline ()
    at /usr/home/sandbox/newusb/src/sys/amd64/amd64/exception.S:521
#20 0x0000000000000000 in ?? ()
#21 0x0000000000000000 in ?? ()
#22 0x0000000000000001 in ?? ()
#23 0x0000000000000000 in ?? ()
#24 0x0000000000000000 in ?? ()
#25 0x0000000000000000 in ?? ()
#26 0x0000000000000000 in ?? ()
#27 0x0000000000000000 in ?? ()
#28 0x0000000000000000 in ?? ()
#29 0x0000000000000000 in ?? ()
#30 0x0000000000000000 in ?? ()
#31 0x0000000000000000 in ?? ()
#32 0x0000000000000000 in ?? ()
#33 0x0000000000000000 in ?? ()
#34 0x0000000000000000 in ?? ()
#35 0x0000000000000000 in ?? ()
#36 0x0000000000000000 in ?? ()
#37 0x0000000000000000 in ?? ()
#38 0x0000000000000000 in ?? ()
#39 0x0000000000000000 in ?? ()
#40 0x0000000000000000 in ?? ()
#41 0x0000000000000000 in ?? ()
#42 0x0000000000000000 in ?? ()
#43 0x0000000000000000 in ?? ()
#44 0x0000000001024000 in ?? ()
#45 0x0000000000000000 in ?? ()
#46 0xffffffff80b8dd70 in sleepq_chains ()
#47 0xffffffff80b756c0 in tdq_cpu ()
#48 0xffffff00024e96e0 in ?? ()
#49 0xfffffffeb7267ac0 in ?? ()
#50 0xfffffffeb7267a78 in ?? ()
#51 0xffffff0002ddd370 in ?? ()
#52 0xffffffff80511f90 in sched_switch (td=0xfffffffe40f840f0, 
    newtd=0xffffffff804377f4, flags=Variable "flags" is not available.
)
    at /usr/home/sandbox/newusb/src/sys/kern/sched_ule.c:1844
Previous frame inner to this frame (corrupt stack?)
(kgdb) 

------------------------------------------------------------------------
ps -axl

  UID   PID  PPID CPU PRI NI   VSZ   RSS MWCHAN STAT  TT       TIME COMMAND
    0     0     0   0 -68  0     0     0 -      DLs   ??  747427:30.00 [kernel]
    0     1     0   0  44  0  2180     0 wait   DLs   ??  5472248:30.00 [init]
    0     2     0   0  -8  0     0     0 -      DL    ??  1887795:30.00 [g_event]
    0     3     0   0  -8  0     0     0 -      DL    ??  159170669:00.00 [g_up]
    0     4     0   0  -8  0     0     0 -      DL    ??  80623968:30.00 [g_down]
    0     5     0   0 -16  0     0     0 tq->tq DL    ??    0:00.00 [system_tas
    0     6     0   0 -16  0     0     0 tq->tq DL    ??    0:00.00 [system_tas
    0     7     0   0 -16  0     0     0 tq->tq DL    ??    0:00.00 [system_tas
    0     8     0   0 -16  0     0     0 tq->tq DL    ??    0:00.00 [system_tas
    0     9     0   0 -16  0     0     0 ccb_sc DL    ??    0:00.00 [xpt_thrd]
    0    10     0   0 -16  0     0     0 audit_ DL    ??  5455:30.00 [audit]
    0    11     0   0 171  0     0     0 -      RL    ??  10883646524:50.00 [idle]
    0    12     0   0 -48  0     0     0 -      WL    ??  74210281:20.00 [intr]
    0    13     0   0  44  0     0     0 -      DL    ??  2767776:50.00 [yarrow]
    0    14     0   0 -64  0     0     0 WMSG   DL    ??    0:00.00 [USBPROC]
    0    15     0   0 -64  0     0     0 WMSG   DL    ??  17473:30.00 [USBPROC]
    0    16     0   0 -16  0     0     0 -      DL    ??    0:00.00 [fw0_probe]
    0    17     0   0 -16  0     0     0 waitin DL    ??  1154:00.00 [sctp_itera
    0    19     0   0  44  0     0     0 d      DL    ??  1475436:10.00 [arc_reclai
    0    21     0   0 -68  0     0     0 WMSG   DL    ??    0:00.00 [USBPROC]
    0    22     0   0 -68  0     0     0 WMSG   DL    ??    0:00.00 [USBPROC]
    0    23     0   0 -64  0     0     0 WMSG   DL    ??    0:00.00 [USBPROC]
    0    26     0   0 -68  0     0     0 WMSG   DL    ??  28989:30.00 [USBPROC]
    0    27     0   0 -68  0     0     0 WMSG   DL    ??    0:00.00 [USBPROC]
    0    28     0   0 -64  0     0     0 WMSG   DL    ??  3466420:40.00 [USBPROC]
    0    29     0   0  45  0     0     0 psleep DL    ??  13022:40.00 [pagedaemon
    0    30     0   0  45  0     0     0 psleep DL    ??  581:40.00 [vmdaemon]
    0    31     0   0  76  0     0     0 pgzero DL    ??  665:40.00 [pagezero]
    0    32     0   0  45  0     0     0 psleep DL    ??  28784:30.00 [bufdaemon]
    0    33     0   0  61  0     0     0 syncer DL    ??  134824082:40.00 [syncer]
    0    34     0   0  45  0     0     0 vlruwt DL    ??  68163:10.00 [vnlru]
    0    35     0   0  45  0     0     0 sdflus DL    ??  58308:10.00 [softdepflu
    0    89     0   0 -68  0     0     0 WMSG   DL    ??  10673:10.00 [USBPROC]
    0    90     0   0  45  0     0     0 tq->tq DL    ??  226:50.00 [spa_zio_is
    0    91     0   0  45  0     0     0 tq->tq DL    ??  206:10.00 [spa_zio_is
    0    92     0   0  45  0     0     0 tq->tq DL    ??  408:20.00 [spa_zio_is
    0    93     0   0  45  0     0     0 tq->tq DL    ??  346:50.00 [spa_zio_is
    0    94     0   0  45  0     0     0 tq->tq DL    ??  271:10.00 [spa_zio_in
    0    95     0   0  45  0     0     0 tq->tq DL    ??  310:00.00 [spa_zio_in
    0    96     0   0  45  0     0     0 tq->tq DL    ??  191:00.00 [spa_zio_in
    0    97     0   0  45  0     0     0 tq->tq DL    ??  342:10.00 [spa_zio_in
    0    98     0   0  45  0     0     0 tq->tq DL    ??  692:10.00 [spa_zio_is
    0    99     0   0  45  0     0     0 tq->tq DL    ??  561:50.00 [spa_zio_is
    0   100     0   0  45  0     0     0 tq->tq DL    ??  407:00.00 [spa_zio_is
    0   101     0   0  45  0     0     0 tq->tq DL    ??  384:40.00 [spa_zio_is
    0   102     0   0  44  0     0     0 tq->tq DL    ??  3915100:00.00 [spa_zio_in
    0   103     0   0  52  0     0     0 tq->tq DL    ??  679657:30.00 [spa_zio_in
    0   104     0   0  52  0     0     0 tq->tq DL    ??  724595:40.00 [spa_zio_in
    0   105     0   0  44  0     0     0 tq->tq DL    ??  3595602:50.00 [spa_zio_in
    0   106     0   0  44  0     0     0 tq->tq DL    ??  161446:00.00 [spa_zio_is
    0   107     0   0  44  0     0     0 tq->tq DL    ??  315903:50.00 [spa_zio_is
    0   108     0   0  44  0     0     0 tq->tq DL    ??  438196:30.00 [spa_zio_is
    0   109     0   0  47  0     0     0 tq->tq DL    ??  68604:10.00 [spa_zio_is
    0   110     0   0  44  0     0     0 tq->tq DL    ??  192541000:20.00 [spa_zio_in
    0   111     0   0  44  0     0     0 tq->tq DL    ??  183661574:50.00 [spa_zio_in
    0   112     0   0  44  0     0     0 tq->tq DL    ??  180367061:20.00 [spa_zio_in
    0   113     0   0  48  0     0     0 tq->tq DL    ??  202043190:20.00 [spa_zio_in
    0   114     0   0  45  0     0     0 tq->tq DL    ??  683:50.00 [spa_zio_is
    0   115     0   0  45  0     0     0 tq->tq DL    ??  595:40.00 [spa_zio_is
    0   116     0   0  45  0     0     0 tq->tq DL    ??  486:10.00 [spa_zio_is
    0   117     0   0  45  0     0     0 tq->tq DL    ??  226:50.00 [spa_zio_is
    0   118     0   0  45  0     0     0 tq->tq DL    ??  386:30.00 [spa_zio_in
    0   119     0   0  45  0     0     0 tq->tq DL    ??  790:20.00 [spa_zio_in
    0   120     0   0  45  0     0     0 tq->tq DL    ??  419:40.00 [spa_zio_in
    0   121     0   0  45  0     0     0 tq->tq DL    ??  173:20.00 [spa_zio_in
    0   122     0   0  45  0     0     0 tq->tq DL    ??  771:10.00 [spa_zio_is
    0   123     0   0  45  0     0     0 tq->tq DL    ??  746:30.00 [spa_zio_is
    0   124     0   0  45  0     0     0 tq->tq DL    ??  525:40.00 [spa_zio_is
    0   125     0   0  45  0     0     0 tq->tq DL    ??  736:40.00 [spa_zio_is
    0   126     0   0  45  0     0     0 tq->tq DL    ??  528:40.00 [spa_zio_in
    0   127     0   0  45  0     0     0 tq->tq DL    ??  309:40.00 [spa_zio_in
    0   128     0   0  45  0     0     0 tq->tq DL    ??  246:40.00 [spa_zio_in
    0   129     0   0  45  0     0     0 tq->tq DL    ??  531:10.00 [spa_zio_in
    0   130     0   0  45  0     0     0 tq->tq DL    ??  3007:50.00 [spa_zio_is
    0   131     0   0  45  0     0     0 tq->tq DL    ??  289:40.00 [spa_zio_is
    0   132     0   0  45  0     0     0 tq->tq DL    ??  289:20.00 [spa_zio_is
    0   133     0   0  45  0     0     0 tq->tq DL    ??  432:40.00 [spa_zio_is
    0   134     0   0  45  0     0     0 tq->tq DL    ??  546:50.00 [spa_zio_in
    0   135     0   0  45  0     0     0 tq->tq DL    ??  312:00.00 [spa_zio_in
    0   136     0   0  45  0     0     0 tq->tq DL    ??  319:40.00 [spa_zio_in
    0   137     0   0  45  0     0     0 tq->tq DL    ??  673:40.00 [spa_zio_in
    0   138     0   0  44  0     0     0 ffffff000407c910 DL    ??  7618572:50.00 [vdev:worke
    0   139     0   0 -68  0     0     0 -      RL    ??  907164:40.00 [USBPROC]
    0   140     0   0  44  0     0     0 2      DL    ??  7583535:20.00 [vdev:worke
    0   141     0   0  44  0     0     0 2      DL    ??  7083820:40.00 [vdev:worke
    0   142     0   0  45  0     0     0 tx->tx DL    ??  15303:20.00 [txg_thread
    0   143     0   0  44  0     0     0 tx->tx DL    ??  164076779:50.00 [txg_thread
    0   144     0   0  45  0     0     0 tx->tx DL    ??  9893:50.00 [txg_thread
    0   145     0   0  45  0     0     0 tq->tq DL    ??  746:40.00 [zil_clean]
    0   171     0   0 -68  0     0     0 WMSG   DL    ??  17984:50.00 [USBPROC]
    0   172     0   0 -68  0     0     0 WMSG   DL    ??  2636:50.00 [USBPROC]
    0   191     0   0 -68  0     0     0 WMSG   DL    ??  6680:30.00 [USBPROC]
    0   192     0   0 -68  0     0     0 WMSG   DL    ??  1121246:10.00 [USBPROC]
    0   193     0   0 -68  0     0     0 WMSG   DL    ??  1451850:30.00 [USBPROC]
    0   194     0   0 -64  0     0     0 WMSG   DL    ??  777326:00.00 [USBPROC]
    0   195     0   0 -68  0     0     0 WMSG   DL    ??  8560:40.00 [USBPROC]
    0   196     0   0 -68  0     0     0 WMSG   DL    ??  108571:50.00 [USBPROC]
    0   265     0   0  45  0     0     0 tq->tq DL    ??  913:20.00 [spa_zio_is
    0   266     0   0  45  0     0     0 tq->tq DL    ??  556:20.00 [spa_zio_is
    0   267     0   0  45  0     0     0 tq->tq DL    ??  264:20.00 [spa_zio_is
    0   268     0   0  45  0     0     0 tq->tq DL    ??  738:20.00 [spa_zio_is
    0   269     0   0  45  0     0     0 tq->tq DL    ??  568:00.00 [spa_zio_in
    0   270     0   0  45  0     0     0 tq->tq DL    ??  599:30.00 [spa_zio_in
    0   271     0   0  45  0     0     0 tq->tq DL    ??  841:00.00 [spa_zio_in
    0   272     0   0  45  0     0     0 tq->tq DL    ??  522:30.00 [spa_zio_in
    0   273     0   0  45  0     0     0 tq->tq DL    ??  730:00.00 [spa_zio_is
    0   274     0   0  45  0     0     0 tq->tq DL    ??  495:10.00 [spa_zio_is
    0   275     0   0  45  0     0     0 tq->tq DL    ??  188:10.00 [spa_zio_is
    0   276     0   0  45  0     0     0 tq->tq DL    ??  311:00.00 [spa_zio_is
    0   277     0   0  45  0     0     0 tq->tq DL    ??  99792:30.00 [spa_zio_in
    0   278     0   0  45  0     0     0 tq->tq DL    ??  220438:10.00 [spa_zio_in
    0   279     0   0  45  0     0     0 tq->tq DL    ??  99017:10.00 [spa_zio_in
    0   280     0   0  45  0     0     0 tq->tq DL    ??  234007:10.00 [spa_zio_in
    0   281     0   0  45  0     0     0 tq->tq DL    ??  13354:20.00 [spa_zio_is
    0   282     0   0  45  0     0     0 tq->tq DL    ??  13582:50.00 [spa_zio_is
    0   283     0   0  45  0     0     0 tq->tq DL    ??  26698:20.00 [spa_zio_is
    0   284     0   0  45  0     0     0 tq->tq DL    ??  26920:10.00 [spa_zio_is
    0   285     0   0  45  0     0     0 tq->tq DL    ??  168799:10.00 [spa_zio_in
    0   286     0   0  45  0     0     0 tq->tq DL    ??  166680:30.00 [spa_zio_in
    0   287     0   0  45  0     0     0 tq->tq DL    ??  159190:00.00 [spa_zio_in
    0   288     0   0  45  0     0     0 tq->tq DL    ??  157914:20.00 [spa_zio_in
    0   289     0   0  45  0     0     0 tq->tq DL    ??  687:20.00 [spa_zio_is
    0   290     0   0  45  0     0     0 tq->tq DL    ??  890:00.00 [spa_zio_is
    0   291     0   0  45  0     0     0 tq->tq DL    ??  506:30.00 [spa_zio_is
    0   292     0   0  45  0     0     0 tq->tq DL    ??  143:20.00 [spa_zio_is
    0   293     0   0  45  0     0     0 tq->tq DL    ??  766:30.00 [spa_zio_in
    0   294     0   0  45  0     0     0 tq->tq DL    ??  753:10.00 [spa_zio_in
    0   295     0   0  45  0     0     0 tq->tq DL    ??  531:00.00 [spa_zio_in
    0   296     0   0  45  0     0     0 tq->tq DL    ??  424:20.00 [spa_zio_in
    0   297     0   0  45  0     0     0 tq->tq DL    ??  902:40.00 [spa_zio_is
    0   298     0   0  45  0     0     0 tq->tq DL    ??  717:30.00 [spa_zio_is
    0   299     0   0  45  0     0     0 tq->tq DL    ??  538:30.00 [spa_zio_is
    0   300     0   0  45  0     0     0 tq->tq DL    ??  770:40.00 [spa_zio_is
    0   301     0   0  45  0     0     0 tq->tq DL    ??  831:10.00 [spa_zio_in
    0   302     0   0  45  0     0     0 tq->tq DL    ??  876:00.00 [spa_zio_in
    0   303     0   0  45  0     0     0 tq->tq DL    ??  1040:30.00 [spa_zio_in
    0   304     0   0  45  0     0     0 tq->tq DL    ??  966:00.00 [spa_zio_in
    0   305     0   0  45  0     0     0 tq->tq DL    ??  882:00.00 [spa_zio_is
    0   306     0   0  45  0     0     0 tq->tq DL    ??  714:30.00 [spa_zio_is
    0   307     0   0  45  0     0     0 tq->tq DL    ??  905:20.00 [spa_zio_is
    0   308     0   0  45  0     0     0 tq->tq DL    ??  774:30.00 [spa_zio_is
    0   309     0   0  45  0     0     0 tq->tq DL    ??  1262:40.00 [spa_zio_in
    0   310     0   0  45  0     0     0 tq->tq DL    ??  732:50.00 [spa_zio_in
    0   311     0   0  45  0     0     0 tq->tq DL    ??  624:10.00 [spa_zio_in
    0   312     0   0  45  0     0     0 tq->tq DL    ??  769:00.00 [spa_zio_in
    0   313     0   0  45  0     0     0 a      DL    ??  138498:00.00 [vdev:worke
    0   314     0   0  45  0     0     0 tx->tx DL    ??  13274:30.00 [txg_thread
    0   315     0   0  45  0     0     0 zio->i DL    ??  592414:00.00 [txg_thread
    0   316     0   0  45  0     0     0 tx->tx DL    ??  14080:20.00 [txg_thread
    0   369     0   0  45  0     0     0 tq->tq DL    ??  406:20.00 [spa_zio_is
    0   370     0   0  45  0     0     0 tq->tq DL    ??  291:50.00 [spa_zio_is
    0   371     0   0  45  0     0     0 tq->tq DL    ??  654:30.00 [spa_zio_is
    0   372     0   0  45  0     0     0 tq->tq DL    ??  515:10.00 [spa_zio_is
    0   373     0   0  45  0     0     0 tq->tq DL    ??  289:50.00 [spa_zio_in
    0   374     0   0  45  0     0     0 tq->tq DL    ??  781:10.00 [spa_zio_in
    0   375     0   0  45  0     0     0 tq->tq DL    ??  689:40.00 [spa_zio_in
    0   376     0   0  45  0     0     0 tq->tq DL    ??  363:40.00 [spa_zio_in
    0   377     0   0  45  0     0     0 tq->tq DL    ??  388:50.00 [spa_zio_is
    0   378     0   0  45  0     0     0 tq->tq DL    ??  577:50.00 [spa_zio_is
    0   379     0   0  45  0     0     0 tq->tq DL    ??  386:20.00 [spa_zio_is
    0   380     0   0  45  0     0     0 tq->tq DL    ??  633:00.00 [spa_zio_is
    0   381     0   0  45  0     0     0 tq->tq DL    ??  461038:00.00 [spa_zio_in
    0   382     0   0  45  0     0     0 tq->tq DL    ??  469792:50.00 [spa_zio_in
    0   383     0   0  45  0     0     0 tq->tq DL    ??  452893:40.00 [spa_zio_in
    0   384     0   0  76  0     0     0 tq->tq DL    ??  260950:00.00 [spa_zio_in
    0   385     0   0  45  0     0     0 tq->tq DL    ??  838:10.00 [spa_zio_is
    0   386     0   0  45  0     0     0 tq->tq DL    ??  359:50.00 [spa_zio_is
    0   387     0   0  44  0     0     0 tq->tq DL    ??  648:20.00 [spa_zio_is
    0   388     0   0  44  0     0     0 tq->tq DL    ??  507:00.00 [spa_zio_is
    0   389     0   0  44  0     0     0 tq->tq DL    ??  717:00.00 [spa_zio_in
    0   390     0   0  45  0     0     0 tq->tq DL    ??  449:30.00 [spa_zio_in
    0   391     0   0  44  0     0     0 tq->tq DL    ??  500:20.00 [spa_zio_in
    0   392     0   0  44  0     0     0 tq->tq DL    ??  351:20.00 [spa_zio_in
    0   393     0   0  45  0     0     0 tq->tq DL    ??  779:40.00 [spa_zio_is
    0   394     0   0  45  0     0     0 tq->tq DL    ??  856:30.00 [spa_zio_is
    0   395     0   0  45  0     0     0 tq->tq DL    ??  602:10.00 [spa_zio_is
    0   396     0   0  45  0     0     0 tq->tq DL    ??  525:00.00 [spa_zio_is
    0   397     0   0  45  0     0     0 tq->tq DL    ??  979:00.00 [spa_zio_in
    0   398     0   0  45  0     0     0 tq->tq DL    ??  821:20.00 [spa_zio_in
    0   399     0   0  45  0     0     0 tq->tq DL    ??  671:00.00 [spa_zio_in
    0   400     0   0  45  0     0     0 tq->tq DL    ??  1013:40.00 [spa_zio_in
    0   401     0   0  45  0     0     0 tq->tq DL    ??  595:30.00 [spa_zio_is
    0   402     0   0  45  0     0     0 tq->tq DL    ??  363:30.00 [spa_zio_is
    0   403     0   0  45  0     0     0 tq->tq DL    ??  444:50.00 [spa_zio_is
    0   404     0   0  45  0     0     0 tq->tq DL    ??  749:20.00 [spa_zio_is
    0   405     0   0  45  0     0     0 tq->tq DL    ??  942:00.00 [spa_zio_in
    0   406     0   0  45  0     0     0 tq->tq DL    ??  379:30.00 [spa_zio_in
    0   407     0   0  45  0     0     0 tq->tq DL    ??  583:00.00 [spa_zio_in
    0   408     0   0  45  0     0     0 tq->tq DL    ??  553:00.00 [spa_zio_in
    0   409     0   0  45  0     0     0 tq->tq DL    ??  570:30.00 [spa_zio_is
    0   410     0   0  45  0     0     0 tq->tq DL    ??  344:10.00 [spa_zio_is
    0   411     0   0  45  0     0     0 tq->tq DL    ??  527:30.00 [spa_zio_is
    0   412     0   0  45  0     0     0 tq->tq DL    ??  449:20.00 [spa_zio_is
    0   413     0   0  45  0     0     0 tq->tq DL    ??  505:00.00 [spa_zio_in
    0   414     0   0  45  0     0     0 tq->tq DL    ??  685:40.00 [spa_zio_in
    0   415     0   0  45  0     0     0 tq->tq DL    ??  467:50.00 [spa_zio_in
    0   416     0   0  45  0     0     0 tq->tq DL    ??  535:00.00 [spa_zio_in
    0   417     0   0  44  0     0     0 d      DL    ??  3205:20.00 [vdev:worke
    0   418     0   0  44  0     0     0 d      DL    ??  3536:10.00 [vdev:worke
    0   419     0   0  44  0     0     0 1      DL    ??  3537:00.00 [vdev:worke
    0   420     0   0  44  0     0     0 1      DL    ??  3025:10.00 [vdev:worke
    0   421     0   0  45  0     0     0 tx->tx DL    ??  13045:50.00 [txg_thread
    0   422     0   0  45  0     0     0 tx->tx DL    ??  24917:00.00 [txg_thread
    0   423     0   0  45  0     0     0 tx->tx DL    ??  10366:50.00 [txg_thread
    0   433     0   0  45  0     0     0 tq->tq DL    ??  622:20.00 [zil_clean]
    0   434     0   0  45  0     0     0 tq->tq DL    ??  4334:00.00 [zil_clean]
    0   435     0   0  45  0     0     0 tq->tq DL    ??  214:20.00 [zil_clean]
    0   448     0   0  45  0     0     0 tq->tq DL    ??  218:20.00 [zil_clean]
    0   455     0   0  45  0     0     0 tq->tq DL    ??  10766862:50.00 [zil_clean]
    0   545     1   0  76  0  2600     0 pause  Ds    ??  20311:10.00 [adjkerntz]
    0  1072     1   0  76  0  2176     0 select Ds    ??  26750:00.00 [devd]
    0  1315     1   0  44  0  5724     0 select Ds    ??  2184241:50.00 [syslogd]
    0  1370     1   0  76  0  5744     0 auditd Ds    ??  425594:30.00 [auditd]
    0  1503     1   0  44  0  5724     0 select Ds    ??  345377:40.00 [powerd]
65534  1558     1   0  44  0  4640     0 select Ds    ??  329227:40.00 [mdnsd]
    0  1571     1   0  61  0 21812     0 kqread Ds    ??  1627382:50.00 [cupsd]
    1  1580     1   0  44  0  7236     0 select Ds    ??  73565:50.00 [slpd]
    0  1612     1   0  76  0 24684     0 select Ds    ??  165899:50.00 [sshd]
    0  1617     1   0  44  0 10692     0 select Ds    ??  390314:20.00 [sendmail]
   25  1623     1   0  76  0 10692     0 pause  Ds    ??  444533:40.00 [sendmail]
    0  1629     1   0  44  0  6780     0 nanslp Ds    ??  308518:30.00 [cron]
    0  1638     1   0  44  0  5672     0 nanslp Ds    ??  45776:40.00 [uptimed]
    0  1650     1   0  76  0  6872     0 select Ds    ??  127881:40.00 [moused]
    0  1674     1   0  76  0  7832     0 select Ds    ??  207752:10.00 [inetd]
    0  1700     1   0  44  0 20476     0 wait   Ds    ??  457620:30.00 [login]
    0  1701     1   0  44  0 20476     0 wait   Ds    ??  232783:10.00 [login]
    0  1702     1   0  76  0  5720     0 tty in Ds+   ??  195147:40.00 [getty]
    0  1703     1   0  76  0  5720     0 tty in Ds+   ??  190936:00.00 [getty]
    0  1704     1   0  76  0  5720     0 tty in Ds+   ??  374987:20.00 [getty]
    0  1705     1   0  76  0  5720     0 tty in Ds+   ??  611203:40.00 [getty]
    0  1706     1   0  76  0  5720     0 tty in Ds+   ??  224722:50.00 [getty]
    0  1707     1   0  76  0  5720     0 tty in Ds+   ??  203391:00.00 [getty]
    0  1708     1   0  76  0  5720     0 tty in Ds+   ??  313967:20.00 [getty]
    0  1709     1   0  76  0  5720     0 tty in Ds+   ??  205962:50.00 [getty]
    0  1710     1   0  76  0  5720     0 tty in Ds+   ??  244910:10.00 [getty]
    0  1714  1700   0  44  0 10132     0 tty in D+    ??  1077786:20.00 [tcsh]
    0  1775     0   0  45  0     0     0 tq->tq DL    ??    0:00.00 [zil_clean]
    0  1777     0   0  45  0     0     0 tq->tq DL    ??    0:00.00 [zil_clean]
    0  1778     0   0  45  0     0     0 tq->tq DL    ??    0:00.00 [zil_clean]
    0  1939     0   0  45  0     0     0 tq->tq DL    ??    0:00.00 [zil_clean]
    0  1940     0   0  45  0     0     0 tq->tq DL    ??    0:00.00 [zil_clean]
    0  1941     0   0  45  0     0     0 tq->tq DL    ??    0:00.00 [zil_clean]
    0  1942     0   0  45  0     0     0 tq->tq DL    ??    0:00.00 [zil_clean]
    0  1943     0   0  45  0     0     0 tq->tq DL    ??    0:00.00 [zil_clean]
    0  1944     0   0  45  0     0     0 tq->tq DL    ??    0:00.00 [zil_clean]
    0  1945     0   0  45  0     0     0 tq->tq DL    ??    0:00.00 [zil_clean]
    0  1946     0   0  45  0     0     0 tq->tq DL    ??    0:00.00 [zil_clean]
    0  1947     0   0  45  0     0     0 tq->tq DL    ??    0:00.00 [zil_clean]
    0  1948     0   0  45  0     0     0 tq->tq DL    ??    0:00.00 [zil_clean]
    0  1949     0   0  45  0     0     0 tq->tq DL    ??    0:00.00 [zil_clean]
    0  1950     0   0  45  0     0     0 tq->tq DL    ??    0:00.00 [zil_clean]
    0  1951     0   0  45  0     0     0 tq->tq DL    ??    0:00.00 [zil_clean]
    0  1952     0   0  45  0     0     0 tq->tq DL    ??    0:00.00 [zil_clean]
    0  1953     0   0  45  0     0     0 tq->tq DL    ??    0:00.00 [zil_clean]
    0  1954     0   0  45  0     0     0 tq->tq DL    ??    0:00.00 [zil_clean]
    0  1955     0   0  45  0     0     0 tq->tq DL    ??    0:00.00 [zil_clean]
    0  1956     0   0  45  0     0     0 tq->tq DL    ??    0:00.00 [zil_clean]
    0  1957     0   0  45  0     0     0 tq->tq DL    ??    0:00.00 [zil_clean]
    0  1958     0   0  45  0     0     0 tq->tq DL    ??    0:00.00 [zil_clean]
    0  1960     0   0  45  0     0     0 tq->tq DL    ??    0:00.00 [zil_clean]
    0  1962     0   0  45  0     0     0 tq->tq DL    ??    0:00.00 [zil_clean]
    0  1964     0   0  45  0     0     0 tq->tq DL    ??    0:00.00 [zil_clean]
    0  1966     0   0  45  0     0     0 tq->tq DL    ??    0:00.00 [zil_clean]
    0  1967     0   0  45  0     0     0 tq->tq DL    ??    0:00.00 [zil_clean]
    0  1997  1701   0  44  0 10132     0 pause  D     ??  184970:50.00 [tcsh]
    0  2001  1997   0  45  0 14452     0 tx->tx D+    ??    0:00.00 [zpool]

------------------------------------------------------------------------
vmstat -s

        0 cpu context switches
        0 device interrupts
        0 software interrupts
        0 traps
        0 system calls
        0 kernel threads created
        0  fork() calls
        0 vfork() calls
        0 rfork() calls
        0 swap pager pageins
        0 swap pager pages paged in
        0 swap pager pageouts
        0 swap pager pages paged out
        0 vnode pager pageins
        0 vnode pager pages paged in
        0 vnode pager pageouts
        0 vnode pager pages paged out
        0 page daemon wakeups
        0 pages examined by the page daemon
     2934 pages reactivated
        0 copy-on-write faults
        0 copy-on-write optimized faults
        0 zero fill pages zeroed
        0 zero fill pages prezeroed
        0 intransit blocking page faults
        0 total VM faults taken
        0 pages affected by kernel thread creation
        0 pages affected by  fork()
        0 pages affected by vfork()
        0 pages affected by rfork()
     3251 pages cached
        0 pages freed
        0 pages freed by daemon
    83854 pages freed by exiting processes
     5662 pages active
     2107 pages inactive
      109 pages in VM cache
   250660 pages wired down
   745813 pages free
     4096 bytes per page
    78523 total name lookups
          cache hits (86% pos + 3% neg) system 0% per-directory
          deletions 0%, falsehits 0%, toolong 0%

------------------------------------------------------------------------
vmstat -m

         Type InUse MemUse HighUse Requests  Size(s)
  ntfs_nthash    -1  -511K       -        0  
    pfs_nodes   -20    -4K       -        0  
  pfs_vncache    -1     0K       -        0  
      fw_xfer   255     0K       -      256  16
     firewire  4082   -37K       -     4096  16,32
         GEOM 2301117 -2321K       -  2303408  16,32,64,128,256,512,1024,2048,4096,8192
       isadev    -8     0K       -        0  
      CAM XPT 40167   -58K       -    40320  32,64,128,256,512
     pci_link   -68    -4K       -        0  
         cdev   -12    -2K       -        0  
          sbp   -96   -12K       -        0  
  ddb_capture    -1   -47K       -        0  
        sigio    -1     0K       -        0  
     filedesc 890917 -1005K       -   892928  16,32,64,128,256,512,1024,2048,4096,8192
         kenv   -25    -9K       -       48  32
       kqueue 32226   -32K       -    32256  64,128,256
    proc-args 42410   -42K       -    43168  16,32,64,128,256,512,1024,2048,4096
      ithread   -77   -11K       -        0  
       prison    -1     0K       -        0  
       KTRACE  -100   -11K       -        0  
       linker  1355  -160K       -     1568  16,32,64,128,512
        lockf   418    -2K       -      448  64
       ip6ndp    -6     0K       -        0  
         temp 125335915 -122423K       - 125343008  16,32,64,128,256,512,1024,2048,4096,8192,16384,32768
       devbuf 361425 -13088K       -   367648  16,32,128,256,512,1024,4096
       module  -431   -52K       -        0  
     mtx_pool    -1    -7K       -        0  
      subproc 7141528 -8238K       -  7143936  16,32,64,128,256,512,1024,2048,4096,8192
         proc    -2   -15K       -        0  
      session   608    -3K       -      640  16,32
         pgrp  5558    -8K       -     5632  16,32,128,256
         cred 1585026 -1566K       -  1591296  16,32,64,128,256,512,1024,2048,4096,8192,16384,32768
      uidinfo   184    -1K       -      192  16,32
       plimit 60926   -63K       -    61184  16,32,64,128,256,512,1024
    sysctltmp 14679   -13K       -    15024  16,32,64,128,512,2048
    sysctloid  2500  -217K       -     7056  16,32,64,128,256,1024,2048
       sysctl 15292   -14K       -    15920  32,64,128,256,1024,4096
      callout    -3 -1535K       -        0  
         umtx  -452   -55K       -        0  
     p1003.1b    -1     0K       -        0  
         SWAP    -8 -8751K       -        0  
       bus-sc 25513187 -25071K       - 25516768  16,32,128,256,512,1024,2048,16384,32768
          bus 1324171 -1395K       -  1330928  16,32,64,128,256,512,1024,2048,4096,8192,65536
        clist   -54    -5K       -        0  
      devstat   -26   -51K       -        0  
 eventhandler   -66    -4K       -        0  
       kbdmux    -6    -7K       -        0  
         kobj 454261 -1579K       -   454656  16,32,64,128,256,1024
         rman 54940   -74K       -    55552  32,256,512,2048,4096
         sbuf 1113064 -1086K       -  1113920  16,32,64,128,256,512,1024,2048,4096
        stack   510     0K       -      512  32
    taskqueue   -13     0K       -        0  
       Unitno   301     0K       -      320  32,64
       acpica 3955121 -4083K       -  3991088  16,64,256,512,1024,16384,32768,65536,131072,262144
     acpitask    63     0K       -       64  16
CAM dev queue    -4     0K       -        0  
      Witness    -1  -127K       -        0  
          iov 217186  -211K       -   218080  32,64,128,256,1024,2048,8192
       select   -20    -1K       -        0  
     ioctlops 6375330 -6235K       -  6382160  16,64,128,256,512,1024,2048,32768,65536
          msg    -4   -29K       -        0  
          sem    -4    -6K       -        0  
          shm    -1   -15K       -        0  
          tty   -21   -20K       -        0  
     mbuf_tag   529     0K       -      544  16,32,128
        shmfd    -1    -7K       -        0  
    CAM queue  2445    -1K       -     2560  16,32,64,256,512
          pcb  1743  -157K       -     1824  16,32,64,128,512
       soname 10028    -9K       -    10384  16,32,64,128,256,512,1024,2048
     vfscache    -1 -1023K       -        0  
     vfs_hash    -1  -511K       -        0  
       vnodes    -2     0K       -        0  
      acpisem   -15     0K       -        0  
  vnodemarker 1129310 -1104K       -  1131520  32,64,128,256,512,1024,2048,4096,8192,16384
        mount 54510   -71K       -    56576  32,64,128,512,1024,2048,4096,16384
          BPF    -6     0K       -        0  
  ether_multi   764    -1K       -      816  16,32,64,128
       ifaddr  3654   -21K       -     3968  64,128,256,512,1024
        ifnet    -6    -9K       -        0  
        clone    -5   -19K       -        0  
       arpcom    -2     0K       -        0  
       fw_com    -1     0K       -        0  
  ata_generic    -6    -5K       -        0  
    ad_driver    -4     0K       -        0  
     routetbl 13587   -15K       -    13664  16,32,64,256,512
     80211vap    -1    -3K       -        0  
  80211crypto    -4     0K       -        0  
     80211com    -1    -7K       -        0  
  80211nodeie  1275    -1K       -     1280  16,32
    80211node 12286   -23K       -    12288  16
    80211scan    -3    -5K       -        0  
         igmp    -1     0K       -        0  
   in_msource  4089    -6K       -     4096  64
  ip_moptions  1266    -1K       -     1280  128
     in_multi    -4     0K       -        0  
    sctp_iter   510     0K       -      512  32
     sctp_ifn    -2     0K       -        0  
     sctp_ifa    -3     0K       -        0  
     sctp_vrf    -1     0K       -        0  
    sctp_a_it    30     0K       -       32  32
    hostcache    -1   -27K       -        0  
    ar_driver 69612   -67K       -    69632  64,256
     syncache    -1   -91K       -        0  
    in6_multi   -16     0K       -        0  
   acd_driver    -2    -3K       -        0  
  nfss_daemon    -1   -15K       -        0  
   audit_data   508     0K       -      512  16,32
audit_evclass  5750   -21K       -     6464  32,128,1024,2048
       newblk    -1     0K       -        0  
     inodedep    -1  -511K       -        0  
      pagedep    -1  -127K       -        0  
  ufs_dirhash    -3     0K       -        0  
    ufs_mount    -3    -5K       -        0  
      UMAHash 19446   -38K       -    19456  32,64
    vm_pgdata    -5  -127K       -        0  
      scsi_da   705     0K       -      752  16,32,64,128,256
      CAM SIM    -4     0K       -        0  
      entropy -1024   -63K       -        0  
      io_apic    -1    -1K       -        0  
   CAM periph  3377    -5K       -     3408  16,32,64,128
      memdesc    -1    -3K       -        0  
      acpidev   -69    -3K       -        0  
     nexusdev    -3     0K       -        0  
         UART    -3     0K       -        0  
     atkbddev    -2     0K       -        0  
       USBdev  2033   -13K       -     2048  32
          USB 98263  -251K       -    98304  32,64,128
       DEVFS1   894   -64K       -     1024  32
       DEVFS3   623   -35K       -      768  16,32
        DEVFS    -2     0K       -       16  16
      solaris 9120189812 -9801837K       - 9128038272  16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536,131072,262144,524288,1048576,2097152,4194304,8388608,16777216,33554432
   kstat_data    -1     0K       -        0  
        linux   -12     0K       -        0  

------------------------------------------------------------------------
vmstat -z

ITEM                     SIZE     LIMIT      USED      FREE  REQUESTS  FAILURES

UMA Kegs:                 200,        0,       93,        9,       93,        0
UMA Zones:                280,        0,       93,       11,       93,        0
UMA Slabs:                128,        0,    12645,       57,    99014,        0
UMA RCntSlabs:            128,        0,      279,       11,      279,        0
UMA Hash:                 256,        0,        6,        9,        8,        0
16 Bucket:                152,        0,      125,        0,      125,        0
32 Bucket:                280,        0,      117,        9,      117,        0
64 Bucket:                536,        0,      147,        0,      147,      658
128 Bucket:              1048,        0,     1126,        2,     1126,     1817
VM OBJECT:                200,        0,     1347,      325,    25494,        0
MAP:                      216,        0,        7,       29,        7,        0
KMAP ENTRY:               112,   150216,      528,      462,   240659,        0
MAP ENTRY:                112,        0,      841,      446,    57149,        0
DP fakepg:                112,        0,        0,        0,        0,        0
mt_zone:                 2056,        0,      262,        1,      262,        0
16:                        16,        0,     3340,      524,   734431,        0
32:                        32,        0,     3692,    12266,  3883221,        0
64:                        64,        0,    10658,     4798,  1238880,        0
128:                      128,        0,    17006,      481,  1301704,        0
256:                      256,        0,      930,    90600,   626721,        0
512:                      512,        0,     2572,      228,    24619,        0
1024:                    1024,        0,      203,     1561,    19987,        0
2048:                    2048,        0,      598,      196,     5822,        0
4096:                    4096,        0,      756,      395,    13709,        0
Files:                     80,        0,      100,      260,     8865,        0
TURNSTILE:                136,        0,      453,       67,      453,        0
umtx pi:                   96,        0,        0,        0,        0,        0
PROC:                    1072,        0,      265,      131,     2009,        0
THREAD:                   872,        0,      421,       31,      421,        0
SLEEPQUEUE:                80,        0,      453,      132,      453,        0
VMSPACE:                  376,        0,       30,       90,     1555,        0
cpuset:                    72,        0,        2,       98,        2,        0
audit_record:             984,        0,        0,       28,        4,        0
mbuf_packet:              256,        0,        0,      512,     9789,        0
mbuf:                     256,        0,        1,      522,      870,        0
mbuf_cluster:            2048,    25600,      512,       46,      512,        0
mbuf_jumbo_page:         4096,    12800,        0,        0,        0,        0
mbuf_jumbo_9k:           9216,    19200,        0,        0,        0,        0
mbuf_jumbo_16k:         16384,    12800,        0,        0,        0,        0
mbuf_ext_refcnt:            4,        0,        0,        0,        0,        0
ACL UMA zone:             388,        0,        0,       30,      124,        0
g_bio:                    216,        0,       16,      704,   367222,        0
ttyinq:                   160,        0,      720,      216,     2520,        0
ttyoutq:                  256,        0,      372,      138,     1302,        0
ata_request:              312,        0,        1,      380,   155536,        0
ata_composite:            336,        0,        0,        0,        0,        0
taskq_ent_cache:           64,        0,     1872,      368,     3772,        0
taskq_cache:              288,        0,       71,       46,      157,        0
VNODE:                    472,        0,      850,      126,     1512,        0
VNODEPOLL:                104,        0,        0,        0,        0,        0
NAMEI:                   1024,        0,        0,       48,    23670,        0
S VFS Cache:              104,        0,      581,      355,     7279,        0
L VFS Cache:              327,        0,        0,       24,        1,        0
DIRHASH:                 1024,        0,       13,        7,       13,        0
NFSMOUNT:                 640,        0,        0,        0,        0,        0
NFSNODE:                  640,        0,        0,        0,        0,        0
pipe:                     712,        0,        2,       63,      880,        0
ksiginfo:                 112,        0,      376,      680,      376,        0
itimer:                   344,        0,        0,       22,        1,        0
KNOTE:                    120,        0,        6,      118,       35,        0
socket:                   640,    12330,       27,       45,      286,        0
unpcb:                    240,    12336,       10,       70,       20,        0
ipq:                       56,      819,        0,        0,        0,        0
udpcb:                    280,    12334,        7,       63,      251,        0
inpcb:                    280,    12334,        9,       33,       13,        0
tcpcb:                    728,    12330,        9,       16,       13,        0
tcptw:                     88,     2478,        0,        0,        0,        0
syncache:                 136,    15372,        0,        0,        0,        0
hostcache:                136,    15372,        0,        0,        0,        0
tcpreass:                  40,     1680,        0,        0,        0,        0
sackhole:                  32,        0,        0,        0,        0,        0
sctp_ep:                 1160,    12330,        0,        0,        0,        0
sctp_asoc:               2152,    40000,        0,        0,        0,        0
sctp_laddr:                48,    80064,        0,      144,        2,        0
sctp_raddr:               584,    80003,        0,        0,        0,        0
sctp_chunk:               136,   400008,        0,        0,        0,        0
sctp_readq:               104,   400032,        0,        0,        0,        0
sctp_stream_msg_out:       88,   400008,        0,        0,        0,        0
sctp_asconf:               40,   400008,        0,        0,        0,        0
sctp_asconf_ack:           48,   400032,        0,        0,        0,        0
ripcb:                    280,    12334,        0,        0,        0,        0
rtentry:                  232,        0,        9,       39,        9,        0
zio_cache:                672,        0,        5,    10585,  1515161,        0
dmu_buf_impl_t:           224,        0,     8767,      260,    40691,        0
dnode_t:                  760,        0,     1578,       97,     2839,        0
arc_buf_hdr_t:            184,        0,    13995,      453,    40842,        0
arc_buf_t:                 40,        0,     8746,      494,    41434,        0
zil_lwb_cache:            200,        0,        2,     2164,     3926,        0
zfs_znode_cache:          336,        0,      673,      218,     1276,        0
selfd:                     56,        0,       44,      271,     3245,        0
SWAPMETA:                 288,   116519,        0,        0,        0,        0
Mountpoints:              800,        0,       37,       43,       88,        0
FFS inode:                176,        0,        7,       56,        7,        0
FFS1 dinode:              128,        0,        0,        0,        0,        0
FFS2 dinode:              256,        0,        7,       38,        7,        0


------------------------------------------------------------------------
vmstat -i

interrupt                          total       rate
irq1: atkbd0                        3006          4
irq12: psm0                            9          0
irq18: fwohci0                         3          0
irq20: atapci2                       117          0
irq21: ohci0+                      58425         89
irq22: ehci0                       19116         29
irq23: atapci1                     72123        109
cpu0: timer                      1695176       2584
cpu1: timer                      1685074       2568
cpu2: timer                      1672747       2549
cpu3: timer                      1672756       2549
Total                            6878552      10485

------------------------------------------------------------------------
pstat -T

100/12328 files
0M/65535M swap space

------------------------------------------------------------------------
pstat -s

Device          1K-blocks     Used    Avail Capacity
/dev/ad4s1b      16777088        0 16777088     0%
/dev/ad6s1d      16777088        0 16777088     0%
/dev/ad12s1d     16777088        0 16777088     0%
/dev/ad14s1b     16777088        0 16777088     0%
Total            67108352        0 67108352     0%

------------------------------------------------------------------------
iostat

iostat: kvm_read(_tk_nin): invalid address (0x0)
iostat: disabling TTY statistics
iostat: kvm_getcptime: invalid address (0x0)
iostat: disabling CPU time statistics
             ad4              ad6              ad12 
  KB/t tps  MB/s   KB/t tps  MB/s   KB/t tps  MB/s 
 56.47  62  3.44  46.83  48  2.18  47.01  48  2.18 

------------------------------------------------------------------------
ipcs -a

Message Queues:
T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP                 CBYTES                 QNUM               QBYTES        LSPID        LRPID STIME    RTIME    CTIME   

Shared Memory:
T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP         NATTCH        SEGSZ         CPID         LPID ATIME    DTIME    CTIME   

Semaphores:
T           ID          KEY MODE        OWNER    GROUP    CREATOR  CGROUP          NSEMS OTIME    CTIME   


------------------------------------------------------------------------
ipcs -T

msginfo:
	msgmax:        16384	(max characters in a message)
	msgmni:           40	(# of message queues)
	msgmnb:         2048	(max characters in a message queue)
	msgtql:           40	(max # of messages in system)
	msgssz:            8	(size of a message segment)
	msgseg:         2048	(# of message segments in system)

shminfo:
	shmmax:     33554432	(max shared memory segment size)
	shmmin:            1	(min shared memory segment size)
	shmmni:          192	(max number of shared memory identifiers)
	shmseg:          128	(max shared memory segments per process)
	shmall:         8192	(max amount of shared memory in pages)

seminfo:
	semmap:           30	(# of entries in semaphore map)
	semmni:           10	(# of semaphore identifiers)
	semmns:           60	(# of semaphores in system)
	semmnu:           30	(# of undo structures in system)
	semmsl:           60	(max # of semaphores per id)
	semopm:          100	(max # of operations per semop call)
	semume:           10	(max # of undo entries per process)
	semusz:          104	(size in bytes of undo structure)
	semvmx:        32767	(semaphore maximum value)
	semaem:        16384	(adjust on exit max value)


------------------------------------------------------------------------
nfsstat

Client Info:
Rpc Counts:
  Getattr   Setattr    Lookup  Readlink      Read     Write    Create    Remove
        0         0         0         0         0         0         0         0
   Rename      Link   Symlink     Mkdir     Rmdir   Readdir  RdirPlus    Access
        0         0         0         0         0         0         0         0
    Mknod    Fsstat    Fsinfo  PathConf    Commit
        0         0         0         0         0
Rpc Info:
 TimedOut   Invalid X Replies   Retries  Requests
        0         0         0         0         0
Cache Info:
Attr Hits    Misses Lkup Hits    Misses BioR Hits    Misses BioW Hits    Misses
        0         0         0         0         0         0         0         0
BioRLHits    Misses BioD Hits    Misses DirE Hits    Misses
        0         0         0         0         0         0

Server Info:
  Getattr   Setattr    Lookup  Readlink      Read     Write    Create    Remove
        0         0         0         0         0         0         0         0
   Rename      Link   Symlink     Mkdir     Rmdir   Readdir  RdirPlus    Access
        0         0         0         0         0         0         0         0
    Mknod    Fsstat    Fsinfo  PathConf    Commit
        0         0         0         0         0
Server Ret-Failed
                0
Server Faults
            0
Server Cache Stats:
   Inprog      Idem  Non-idem    Misses
        0         0         0         0
Server Write Gathering:
 WriteOps  WriteRPC   Opsaved
        0         0         0

------------------------------------------------------------------------
netstat -s

tcp:
	0 packets sent
		0 data packets (0 bytes)
		0 data packets (0 bytes) retransmitted
		0 data packets unnecessarily retransmitted
		0 resends initiated by MTU discovery
		0 ack-only packets (0 delayed)
		0 URG only packets
		0 window probe packets
		0 window update packets
		0 control packets
	0 packets received
		0 acks (for 0 bytes)
		0 duplicate acks
		0 acks for unsent data
		0 packets (0 bytes) received in-sequence
		0 completely duplicate packets (0 bytes)
		0 old duplicate packets
		0 packets with some dup. data (0 bytes duped)
		0 out-of-order packets (0 bytes)
		0 packets (0 bytes) of data after window
		0 window probes
		0 window update packets
		0 packets received after close
		0 discarded for bad checksums
		0 discarded for bad header offset fields
		0 discarded because packet too short
		0 discarded due to memory problems
	0 connection requests
	0 connection accepts
	0 bad connection attempts
	0 listen queue overflows
	0 ignored RSTs in the windows
	0 connections established (including accepts)
	4 connections closed (including 0 drops)
		0 connections updated cached RTT on close
		0 connections updated cached RTT variance on close
		0 connections updated cached ssthresh on close
	0 embryonic connections dropped
	0 segments updated rtt (of 0 attempts)
	0 retransmit timeouts
		0 connections dropped by rexmit timeout
	0 persist timeouts
		0 connections dropped by persist timeout
	0 Connections (fin_wait_2) dropped because of timeout
	0 keepalive timeouts
		0 keepalive probes sent
		0 connections dropped by keepalive
	0 correct ACK header predictions
	0 correct data packet header predictions
	0 syncache entries added
		0 retransmitted
		0 dupsyn
		0 dropped
		0 completed
		0 bucket overflow
		0 cache overflow
		0 reset
		0 stale
		0 aborted
		0 badack
		0 unreach
		0 zone failures
	0 cookies sent
	0 cookies received
	0 SACK recovery episodes
	0 segment rexmits in SACK recovery episodes
	0 byte rexmits in SACK recovery episodes
	0 SACK options (SACK blocks) received
	0 SACK options (SACK blocks) sent
	0 SACK scoreboard overflow
	0 packets with ECN CE bit set
	0 packets with ECN ECT(0) bit set
	0 packets with ECN ECT(1) bit set
	0 successful ECN handshakes
	0 times ECN reduced the congestion window
udp:
	132 datagrams received
	0 with incomplete header
	0 with bad data length field
	0 with bad checksum
	0 with no checksum
	0 dropped due to no socket
	24 broadcast/multicast datagrams undelivered
	0 dropped due to full socket buffers
	0 not for hashed pcb
	108 delivered
	100 datagrams output
	0 times multicast source filter matched
ip:
	1419 total packets received
	0 bad header checksums
	0 with size smaller than minimum
	0 with data size < data length
	0 with ip length > max ip packet size
	0 with header length < data size
	0 with data length < header length
	0 with bad options
	0 with incorrect version number
	0 fragments received
	0 fragments dropped (dup or out of space)
	0 fragments dropped after timeout
	0 packets reassembled ok
	132 packets for this host
	30 packets for unknown/unsupported protocol
	0 packets forwarded (0 packets fast forwarded)
	0 packets not forwardable
	1257 packets received for unknown multicast group
	0 redirects sent
	121 packets sent from this host
	0 packets sent with fabricated ip header
	0 output packets dropped due to no bufs, etc.
	0 output packets discarded due to no route
	0 output datagrams fragmented
	0 fragments created
	0 datagrams that can't be fragmented
	0 tunneling packets that can't find gif
	0 datagrams with bad address in header
icmp:
	0 calls to icmp_error
	0 errors not generated in response to an icmp message
	0 messages with bad code fields
	0 messages less than the minimum length
	0 messages with bad checksum
	0 messages with bad length
	0 multicast echo requests ignored
	0 multicast timestamp requests ignored
	0 message responses generated
	0 invalid return addresses
	0 no return routes
igmp:
	30 messages received
	0 messages received with too few bytes
	0 messages received with bad checksum
	14 membership queries received
	0 membership queries received with invalid field(s)
	16 membership reports received
	0 membership reports received with invalid field(s)
	16 membership reports received for groups to which we belong
	21 membership reports sent
ip6:
	10 total packets received
	0 with size smaller than minimum
	0 with data size < data length
	0 with bad options
	0 with incorrect version number
	0 fragments received
	0 fragments dropped (dup or out of space)
	0 fragments dropped after timeout
	0 fragments that exceeded limit
	0 packets reassembled ok
	0 packets for this host
	0 packets forwarded
	10 packets not forwardable
	0 redirects sent
	4 packets sent from this host
	0 packets sent with fabricated ip header
	0 output packets dropped due to no bufs, etc.
	0 output packets discarded due to no route
	0 output datagrams fragmented
	0 fragments created
	0 datagrams that can't be fragmented
	0 packets that violated scope rules
	10 multicast packets which we don't join
	Input histogram:
		UDP: 10
	Mbuf statistics:
		0 one mbuf
		10 one ext mbuf
		0 two or more ext mbuf
	0 packets whose headers are not continuous
	0 tunneling packets that can't find gif
	0 packets discarded because of too many headers
	0 failures of source address selection
	0 forward cache hit
	0 forward cache miss
	Source addresses selection rule applied:
		1 first candidate
		1 same address
icmp6:
	0 calls to icmp6_error
	0 errors not generated in response to an icmp6 message
	0 errors not generated because of rate limitation
	Output histogram:
		multicast listener report: 4
	0 messages with bad code fields
	0 messages < minimum length
	0 bad checksums
	0 messages with bad length
	Histogram of error messages to be generated:
		0 no route
		0 administratively prohibited
		0 beyond scope
		0 address unreachable
		0 port unreachable
		0 packet too big
		0 time exceed transit
		0 time exceed reassembly
		0 erroneous header field
		0 unrecognized next header
		0 unrecognized option
		0 redirect
		0 unknown
	0 message responses generated
	0 messages with too many ND options
	0 messages with bad ND options
	0 bad neighbor solicitation messages
	0 bad neighbor advertisement messages
	0 bad router solicitation messages
	0 bad router advertisement messages
	0 bad redirect messages
	0 path MTU changes
rip6:
	0 messages received
	0 checksum calcurations on inbound
	0 messages with bad checksum
	0 messages dropped due to no socket
	0 multicast messages dropped due to no socket
	0 messages dropped due to full socket buffers
	0 delivered
	0 datagrams output

------------------------------------------------------------------------
netstat -m

1/1034/1035 mbufs in use (current/cache/total)
0/558/558/25600 mbuf clusters in use (current/cache/total/max)
0/512 mbuf+clusters out of packet secondary zone in use (current/cache)
0/0/0/12800 4k (page size) jumbo clusters in use (current/cache/total/max)
0/0/0/19200 9k jumbo clusters in use (current/cache/total/max)
0/0/0/12800 16k jumbo clusters in use (current/cache/total/max)
18014398509481983K/1374K/1374K bytes allocated to network (current/cache/total)
0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0/0/0 requests for jumbo clusters denied (4k/9k/16k)
0 requests for sfbufs denied
0 requests for sfbufs delayed
0 requests for I/O initiated by sendfile
0 calls to protocol drain routines

------------------------------------------------------------------------
netstat -id

Name    Mtu Network       Address              Ipkts Ierrs    Opkts Oerrs  Coll Drop
fwe0*  1500 <Link#1>      82:00:00:00:00:00        0     0        0     0     0    0 
fwip0  1500 <Link#2>      80:00:00:00:00:00:00:00:0a:02:ff:fe:00:00:00:00        0     0        0     0     0    0 
lo0   16384 <Link#3>                               0     0        0     0     0    0 
lo0   16384 fe80:3::1     fe80:3::1                0     -        0     -     -    - 
lo0   16384 localhost     ::1                      0     -        0     -     -    - 
lo0   16384 your-net      localhost                0     -        0     -     -    - 
rum0   2290 <Link#4>      00:17:3f:72:40:90        0     0      316     2     0    0 
wlan0  1500 <Link#5>      00:17:3f:72:40:90     1435     0      122     0     0    0 
wlan0  1500 192.168.0.0   w2fzz0vc03             162     -      121     -     -    - 

------------------------------------------------------------------------
netstat -anr

Routing tables

Internet:
Destination        Gateway            Flags    Refs      Use  Netif Expire
default            192.168.0.1        UGS         0       76  wlan0
127.0.0.1          127.0.0.1          UH          0        0    lo0
192.168.0.0/24     link#5             UC          0        0  wlan0
192.168.0.1        00:1c:df:77:6c:f5  UHLW        2       14  wlan0   1251

Internet6:
Destination                       Gateway                       Flags      Netif Expire
::1                               ::1                           UHL         lo0
fe80::%lo0/64                     fe80::1%lo0                   U           lo0
fe80::1%lo0                       link#3                        UHL         lo0
ff01:3::/32                       fe80::1%lo0                   UC          lo0
ff02::%lo0/32                     fe80::1%lo0                   UC          lo0

------------------------------------------------------------------------
netstat -anA

Active UNIX domain sockets
Address  Type   Recv-Q Send-Q    Inode     Conn     Refs  Nextref Addr
ffffff0004decd20 stream      0      0 ffffff00856fb938        0        0        0 /var/run/cups.sock
ffffff0004dedb40 stream      0      0 ffffff0004fa0000        0        0        0 /var/run/mdnsd
ffffff0004ded000 stream      0      0 ffffff0004d123b0        0        0        0 /var/run/devd.pipe
ffffff00857fa870 dgram       0      0        0 ffffff0004dede10        0 ffffff00859693c0
ffffff00859693c0 dgram       0      0        0 ffffff0004dede10        0 ffffff0004deca50
ffffff0004dec960 dgram       0      0        0 ffffff0004ea5000        0        0
ffffff0004deca50 dgram       0      0        0 ffffff0004dede10        0 ffffff0004dedd20
ffffff0004dedd20 dgram       0      0        0 ffffff0004dede10        0        0
ffffff0004dede10 dgram       0      0 ffffff0004fa3938        0 ffffff00857fa870        0 /var/run/logpriv
ffffff0004ea5000 dgram       0      0 ffffff0004fa3b10        0 ffffff0004dec960        0 /var/run/log

------------------------------------------------------------------------
netstat -aL


------------------------------------------------------------------------
fstat

USER     CMD          PID   FD MOUNT      INUM MODE         SZ|DV R/W
root     zpool       2001 root /             3 drwxr-xr-x      32  r
root     zpool       2001   wd /            24 drwxr-xr-x      98  r
root     zpool       2001 text /          1090 -r-xr-xr-x   67152  r
root     zpool       2001    0 /dev         55 crw-------   ttyv1 rw
root     zpool       2001    1 /dev         55 crw-------   ttyv1 rw
root     zpool       2001    2 /dev         55 crw-------   ttyv1 rw
root     zpool       2001    3 /dev         76 crw-rw----     zfs rw
root     zpool       2001    4 /dev         31 crw-rw-rw-    null  r
root     zpool       2001    5 /           745 -rw-r--r--       0  r
root     tcsh        1997 root /             3 drwxr-xr-x      32  r
root     tcsh        1997   wd /            24 drwxr-xr-x      98  r
root     tcsh        1997 text /          1008 -r-xr-xr-x  345232  r
root     tcsh        1997   15 /dev         55 crw-------   ttyv1 rw
root     tcsh        1997   16 /dev         55 crw-------   ttyv1 rw
root     tcsh        1997   17 /dev         55 crw-------   ttyv1 rw
root     tcsh        1997   18 /dev         55 crw-------   ttyv1 rw
root     tcsh        1997   19 /dev         55 crw-------   ttyv1 rw
root     zil_clean   1967 root /             3 drwxr-xr-x      32  r
root     zil_clean   1967   wd /             3 drwxr-xr-x      32  r
root     zil_clean   1966 root /             3 drwxr-xr-x      32  r
root     zil_clean   1966   wd /             3 drwxr-xr-x      32  r
root     zil_clean   1964 root /             3 drwxr-xr-x      32  r
root     zil_clean   1964   wd /             3 drwxr-xr-x      32  r
root     zil_clean   1962 root /             3 drwxr-xr-x      32  r
root     zil_clean   1962   wd /             3 drwxr-xr-x      32  r
root     zil_clean   1960 root /             3 drwxr-xr-x      32  r
root     zil_clean   1960   wd /             3 drwxr-xr-x      32  r
root     zil_clean   1958 root /             3 drwxr-xr-x      32  r
root     zil_clean   1958   wd /             3 drwxr-xr-x      32  r
root     zil_clean   1957 root /             3 drwxr-xr-x      32  r
root     zil_clean   1957   wd /             3 drwxr-xr-x      32  r
root     zil_clean   1956 root /             3 drwxr-xr-x      32  r
root     zil_clean   1956   wd /             3 drwxr-xr-x      32  r
root     zil_clean   1955 root /             3 drwxr-xr-x      32  r
root     zil_clean   1955   wd /             3 drwxr-xr-x      32  r
root     zil_clean   1954 root /             3 drwxr-xr-x      32  r
root     zil_clean   1954   wd /             3 drwxr-xr-x      32  r
root     zil_clean   1953 root /             3 drwxr-xr-x      32  r
root     zil_clean   1953   wd /             3 drwxr-xr-x      32  r
root     zil_clean   1952 root /             3 drwxr-xr-x      32  r
root     zil_clean   1952   wd /             3 drwxr-xr-x      32  r
root     zil_clean   1951 root /             3 drwxr-xr-x      32  r
root     zil_clean   1951   wd /             3 drwxr-xr-x      32  r
root     zil_clean   1950 root /             3 drwxr-xr-x      32  r
root     zil_clean   1950   wd /             3 drwxr-xr-x      32  r
root     zil_clean   1949 root /             3 drwxr-xr-x      32  r
root     zil_clean   1949   wd /             3 drwxr-xr-x      32  r
root     zil_clean   1948 root /             3 drwxr-xr-x      32  r
root     zil_clean   1948   wd /             3 drwxr-xr-x      32  r
root     zil_clean   1947 root /             3 drwxr-xr-x      32  r
root     zil_clean   1947   wd /             3 drwxr-xr-x      32  r
root     zil_clean   1946 root /             3 drwxr-xr-x      32  r
root     zil_clean   1946   wd /             3 drwxr-xr-x      32  r
root     zil_clean   1945 root /             3 drwxr-xr-x      32  r
root     zil_clean   1945   wd /             3 drwxr-xr-x      32  r
root     zil_clean   1944 root /             3 drwxr-xr-x      32  r
root     zil_clean   1944   wd /             3 drwxr-xr-x      32  r
root     zil_clean   1943 root /             3 drwxr-xr-x      32  r
root     zil_clean   1943   wd /             3 drwxr-xr-x      32  r
root     zil_clean   1942 root /             3 drwxr-xr-x      32  r
root     zil_clean   1942   wd /             3 drwxr-xr-x      32  r
root     zil_clean   1941 root /             3 drwxr-xr-x      32  r
root     zil_clean   1941   wd /             3 drwxr-xr-x      32  r
root     zil_clean   1940 root /             3 drwxr-xr-x      32  r
root     zil_clean   1940   wd /             3 drwxr-xr-x      32  r
root     zil_clean   1939 root /             3 drwxr-xr-x      32  r
root     zil_clean   1939   wd /             3 drwxr-xr-x      32  r
root     zil_clean   1778 root /             3 drwxr-xr-x      32  r
root     zil_clean   1778   wd /             3 drwxr-xr-x      32  r
root     zil_clean   1777 root /             3 drwxr-xr-x      32  r
root     zil_clean   1777   wd /             3 drwxr-xr-x      32  r
root     zil_clean   1775 root /             3 drwxr-xr-x      32  r
root     zil_clean   1775   wd /             3 drwxr-xr-x      32  r
root     tcsh        1714 root /             3 drwxr-xr-x      32  r
root     tcsh        1714   wd /            24 drwxr-xr-x      98  r
root     tcsh        1714 text /          1008 -r-xr-xr-x  345232  r
root     tcsh        1714    0 /dev         54 crw-------   ttyv0 rw
root     tcsh        1714    1 /dev         54 crw-------   ttyv0 rw
root     tcsh        1714    2 /dev         54 crw-------   ttyv0 rw
root     tcsh        1714   15 /dev         54 crw-------   ttyv0 rw
root     tcsh        1714   16 /dev         54 crw-------   ttyv0 rw
root     tcsh        1714   17 /dev         54 crw-------   ttyv0 rw
root     tcsh        1714   18 /dev         54 crw-------   ttyv0 rw
root     tcsh        1714   19 /dev         54 crw-------   ttyv0 rw
root     getty       1710 root /             3 drwxr-xr-x      32  r
root     getty       1710   wd /             3 drwxr-xr-x      32  r
root     getty       1710 text /usr       1227 -r-xr-xr-x   27728  r
root     getty       1710    0 /dev         64 crw-------   ttyva rw
root     getty       1710    1 /dev         64 crw-------   ttyva rw
root     getty       1710    2 /dev         64 crw-------   ttyva rw
root     getty       1709 root /             3 drwxr-xr-x      32  r
root     getty       1709   wd /             3 drwxr-xr-x      32  r
root     getty       1709 text /usr       1227 -r-xr-xr-x   27728  r
root     getty       1709    0 /dev         63 crw-------   ttyv9 rw
root     getty       1709    1 /dev         63 crw-------   ttyv9 rw
root     getty       1709    2 /dev         63 crw-------   ttyv9 rw
root     getty       1708 root /             3 drwxr-xr-x      32  r
root     getty       1708   wd /             3 drwxr-xr-x      32  r
root     getty       1708 text /usr       1227 -r-xr-xr-x   27728  r
root     getty       1708    0 /dev         62 crw-------   ttyv8 rw
root     getty       1708    1 /dev         62 crw-------   ttyv8 rw
root     getty       1708    2 /dev         62 crw-------   ttyv8 rw
root     getty       1707 root /             3 drwxr-xr-x      32  r
root     getty       1707   wd /             3 drwxr-xr-x      32  r
root     getty       1707 text /usr       1227 -r-xr-xr-x   27728  r
root     getty       1707    0 /dev         61 crw-------   ttyv7 rw
root     getty       1707    1 /dev         61 crw-------   ttyv7 rw
root     getty       1707    2 /dev         61 crw-------   ttyv7 rw
root     getty       1706 root /             3 drwxr-xr-x      32  r
root     getty       1706   wd /             3 drwxr-xr-x      32  r
root     getty       1706 text /usr       1227 -r-xr-xr-x   27728  r
root     getty       1706    0 /dev         60 crw-------   ttyv6 rw
root     getty       1706    1 /dev         60 crw-------   ttyv6 rw
root     getty       1706    2 /dev         60 crw-------   ttyv6 rw
root     getty       1705 root /             3 drwxr-xr-x      32  r
root     getty       1705   wd /             3 drwxr-xr-x      32  r
root     getty       1705 text /usr       1227 -r-xr-xr-x   27728  r
root     getty       1705    0 /dev         59 crw-------   ttyv5 rw
root     getty       1705    1 /dev         59 crw-------   ttyv5 rw
root     getty       1705    2 /dev         59 crw-------   ttyv5 rw
root     getty       1704 root /             3 drwxr-xr-x      32  r
root     getty       1704   wd /             3 drwxr-xr-x      32  r
root     getty       1704 text /usr       1227 -r-xr-xr-x   27728  r
root     getty       1704    0 /dev         58 crw-------   ttyv4 rw
root     getty       1704    1 /dev         58 crw-------   ttyv4 rw
root     getty       1704    2 /dev         58 crw-------   ttyv4 rw
root     getty       1703 root /             3 drwxr-xr-x      32  r
root     getty       1703   wd /             3 drwxr-xr-x      32  r
root     getty       1703 text /usr       1227 -r-xr-xr-x   27728  r
root     getty       1703    0 /dev         57 crw-------   ttyv3 rw
root     getty       1703    1 /dev         57 crw-------   ttyv3 rw
root     getty       1703    2 /dev         57 crw-------   ttyv3 rw
root     getty       1702 root /             3 drwxr-xr-x      32  r
root     getty       1702   wd /             3 drwxr-xr-x      32  r
root     getty       1702 text /usr       1227 -r-xr-xr-x   27728  r
root     getty       1702    0 /dev         56 crw-------   ttyv2 rw
root     getty       1702    1 /dev         56 crw-------   ttyv2 rw
root     getty       1702    2 /dev         56 crw-------   ttyv2 rw
root     login       1701 root /             3 drwxr-xr-x      32  r
root     login       1701   wd /            24 drwxr-xr-x      98  r
root     login       1701 text /usr        817 -r-sr-xr-x   25400  r
root     login       1701    0 /dev         55 crw-------   ttyv1 rw
root     login       1701    1 /dev         55 crw-------   ttyv1 rw
root     login       1701    2 /dev         55 crw-------   ttyv1 rw
root     login       1701    3 /           751 -rw-------     567  r
root     login       1701    4 /           750 -r--r--r--     652  r
root     login       1701    5 /           746 -rw-------     225  r
root     login       1701    6* local dgram ffffff00857fa870 <-> ffffff0004dede10
root     login       1700 root /             3 drwxr-xr-x      32  r
root     login       1700   wd /            24 drwxr-xr-x      98  r
root     login       1700 text /usr        817 -r-sr-xr-x   25400  r
root     login       1700    0 /dev         54 crw-------   ttyv0 rw
root     login       1700    1 /dev         54 crw-------   ttyv0 rw
root     login       1700    2 /dev         54 crw-------   ttyv0 rw
root     login       1700    3 /           751 -rw-------     567  r
root     login       1700    4 /           750 -r--r--r--     652  r
root     login       1700    5 /           746 -rw-------     225  r
root     login       1700    6* local dgram ffffff00859693c0 <-> ffffff0004dede10
root     inetd       1674 root /             3 drwxr-xr-x      32  r
root     inetd       1674   wd /             3 drwxr-xr-x      32  r
root     inetd       1674 text /usr        191 -r-xr-xr-x   47296  r
root     inetd       1674    0 /dev         31 crw-rw-rw-    null rw
root     inetd       1674    1 /dev         31 crw-rw-rw-    null rw
root     inetd       1674    2 /dev         31 crw-rw-rw-    null rw
root     inetd       1674    3 /var       6228 -rw-------       4  w
root     inetd       1674    4* pipe ffffff0004e71858 <-> ffffff0004e719a8      0 rw
root     inetd       1674    5* pipe ffffff0004e719a8 <-> ffffff0004e71858      0 rw
root     moused      1650 root /             3 drwxr-xr-x      32  r
root     moused      1650   wd /             3 drwxr-xr-x      32  r
root     moused      1650 text /usr        194 -r-xr-xr-x   40080  r
root     moused      1650    0 /dev         31 crw-rw-rw-    null rw
root     moused      1650    1 /dev         31 crw-rw-rw-    null rw
root     moused      1650    2 /dev         31 crw-rw-rw-    null rw
root     moused      1650    3 /dev         46 crw-rw-rw-    psm0 rw
root     moused      1650    4 /dev         70 crw-------  consolectl rw
root     moused      1650    5 /var       6227 -rw-------       4  w
root     uptimed     1638 root /             3 drwxr-xr-x      32  r
root     uptimed     1638   wd /             3 drwxr-xr-x      32  r
root     uptimed     1638 text /usr/local 110466 -r-xr-xr-x   15872  r
root     cron        1629 root /             3 drwxr-xr-x      32  r
root     cron        1629   wd /var         13 drwxr-x---       3  r
root     cron        1629 text /usr        256 -r-xr-xr-x   38608  r
root     cron        1629    0 /dev         31 crw-rw-rw-    null rw
root     cron        1629    1 /dev         31 crw-rw-rw-    null rw
root     cron        1629    2 /dev         31 crw-rw-rw-    null rw
root     cron        1629    3 /var       6224 -rw-------       4  w
smmsp    sendmail    1623 root /             3 drwxr-xr-x      32  r
smmsp    sendmail    1623   wd /var        110 drwxrwx---       3  r
smmsp    sendmail    1623 text /usr       1229 -r-xr-sr-x  680072  r
smmsp    sendmail    1623    0 /dev         31 crw-rw-rw-    null  r
smmsp    sendmail    1623    1 /dev         31 crw-rw-rw-    null  w
smmsp    sendmail    1623    2 /dev         31 crw-rw-rw-    null  w
smmsp    sendmail    1623    3* local dgram ffffff0004dec960 <-> ffffff0004ea5000
root     sendmail    1617 root /             3 drwxr-xr-x      32  r
root     sendmail    1617   wd /var        111 drwxr-xr-x       2  r
root     sendmail    1617 text /usr       1229 -r-xr-sr-x  680072  r
root     sendmail    1617    0 /dev         31 crw-rw-rw-    null  r
root     sendmail    1617    1 /dev         31 crw-rw-rw-    null  w
root     sendmail    1617    2 /dev         31 crw-rw-rw-    null  w
root     sendmail    1617    3* local dgram ffffff0004deca50 <-> ffffff0004dede10
root     sendmail    1617    4* internet stream tcp ffffff0085608888
root     sendmail    1617    5* internet6 stream tcp ffffff00856085b0
root     sendmail    1617    6* internet stream tcp ffffff00856082d8
root     sendmail    1617    7 /var       6223 -rw-------      44  w
root     sshd        1612 root /             3 drwxr-xr-x      32  r
root     sshd        1612   wd /             3 drwxr-xr-x      32  r
root     sshd        1612 text /usr        210 -r-xr-xr-x  241824  r
root     sshd        1612    0 /dev         31 crw-rw-rw-    null rw
root     sshd        1612    1 /dev         31 crw-rw-rw-    null rw
root     sshd        1612    2 /dev         31 crw-rw-rw-    null rw
root     sshd        1612    3* internet6 stream tcp ffffff0085703000
root     sshd        1612    4* internet stream tcp ffffff0085608b60
daemon   slpd        1580 root /             3 drwxr-xr-x      32  r
daemon   slpd        1580   wd /             3 drwxr-xr-x      32  r
daemon   slpd        1580 text /usr/local 110501 -r-xr-xr-x   82784  r
daemon   slpd        1580    3 /var       6120 -rw-------  812302  w
daemon   slpd        1580    4* internet stream tcp ffffff0004f9d888
daemon   slpd        1580    5* internet stream tcp ffffff0004f9d5b0
daemon   slpd        1580    6* internet dgram udp ffffff0004d90af0
daemon   slpd        1580    7* internet dgram udp ffffff00857cf9d8
root     cupsd       1571 root /             3 drwxr-xr-x      32  r
root     cupsd       1571   wd /             3 drwxr-xr-x      32  r
root     cupsd       1571 text /usr/local 110523 -rwxr-xr-x  908330  r
root     cupsd       1571    1 /dev          7 crw-rw-rw-  random  r
root     cupsd       1571    2 /var       6215 -rw-r--r--  1564517 rw
root     cupsd       1571    3* internet stream tcp ffffff00857035b0
root     cupsd       1571    4* internet6 stream tcp ffffff00857032d8
root     cupsd       1571    5* local stream ffffff0004decd20
root     cupsd       1571    6* internet dgram udp ffffff0004d6faf0
root     cupsd       1571    7* pipe ffffff0004472b20 <-> ffffff0004472c70      0 rw
root     cupsd       1571    8* pipe ffffff0004472c70 <-> ffffff0004472b20      0 rw
nobody   mdnsd       1558 root /             3 drwxr-xr-x      32  r
nobody   mdnsd       1558   wd /             3 drwxr-xr-x      32  r
nobody   mdnsd       1558 text /usr/local 110499 -rwxr-xr-x  189869  r
nobody   mdnsd       1558    0 /dev         31 crw-rw-rw-    null rw
nobody   mdnsd       1558    1 /dev         31 crw-rw-rw-    null rw
nobody   mdnsd       1558    2 /dev         31 crw-rw-rw-    null rw
nobody   mdnsd       1558    3* internet dgram udp ffffff0004d90578
nobody   mdnsd       1558    4* internet dgram udp ffffff0004d909d8
nobody   mdnsd       1558    5 /           536 -rw-r--r--      44  r
nobody   mdnsd       1558    6* route raw 0 ffffff0004d6e780
nobody   mdnsd       1558    7* local stream ffffff0004dedb40
nobody   mdnsd       1558    8 /           536 -rw-r--r--      44  r
root     powerd      1503 root /             3 drwxr-xr-x      32  r
root     powerd      1503   wd /             3 drwxr-xr-x      32  r
root     powerd      1503 text /usr        227 -r-xr-xr-x   13488  r
root     powerd      1503    0 /dev         31 crw-rw-rw-    null rw
root     powerd      1503    1 /dev         31 crw-rw-rw-    null rw
root     powerd      1503    2 /dev         31 crw-rw-rw-    null rw
root     powerd      1503    3 /var       4542 -rw-------       4  w
root     auditd      1370 root /             3 drwxr-xr-x      32  r
root     auditd      1370   wd /             3 drwxr-xr-x      32  r
root     auditd      1370 text /usr         77 -r-xr-xr-x   20384  r
root     auditd      1370    0 /dev         31 crw-rw-rw-    null rw
root     auditd      1370    1 /dev         31 crw-rw-rw-    null rw
root     auditd      1370    2 /dev         31 crw-rw-rw-    null rw
root     auditd      1370    3* local dgram ffffff0004dedd20 <-> ffffff0004dede10
root     auditd      1370    4 /var       4538 -rw-r--r--       5  w
root     auditd      1370    5 /dev         19 crw-------   audit  r
root     auditd      1370    6 /           746 -rw-------     225  r
root     auditd      1370    8 /           750 -r--r--r--     652  r
root     syslogd     1315 root /             3 drwxr-xr-x      32  r
root     syslogd     1315   wd /             3 drwxr-xr-x      32  r
root     syslogd     1315 text /usr        242 -r-xr-xr-x   38976  r
root     syslogd     1315    0 /dev         31 crw-rw-rw-    null rw
root     syslogd     1315    1 /dev         31 crw-rw-rw-    null rw
root     syslogd     1315    2 /dev         31 crw-rw-rw-    null rw
root     syslogd     1315    3 /var         46 -rw-------       4  w
root     syslogd     1315    4* local dgram ffffff0004ea5000
root     syslogd     1315    5* local dgram ffffff0004dede10
root     syslogd     1315    6* internet6 dgram udp ffffff0004d6fe38
root     syslogd     1315    7* internet dgram udp ffffff0004d90000
root     syslogd     1315    8 /dev         33 crw-------    klog  r
root     syslogd     1315   10 -         -         bad    -
root     syslogd     1315   11 /var       6284 -rw-r--r--   26174  w
root     syslogd     1315   12 /var       6165 -rw-------    3761  w
root     syslogd     1315   13 /var       6127 -rw-------   51509  w
root     syslogd     1315   14 /var       6167 -rw-r-----    5487  w
root     syslogd     1315   15 /var       6199 -rw-r--r--      82  w
root     syslogd     1315   16 /var       6119 -rw-------      82  w
root     syslogd     1315   17 /var       6147 -rw-------   25357  w
root     syslogd     1315   18 /var       6124 -rw-------   82118  w
root     syslogd     1315   19 /var       6146 -rw-------   49305  w
root     syslogd     1315   20 /var       6198 -rw-------  599426  w
root     syslogd     1315   21 /var       6101 -rw-r-----      82  w
root     syslogd     1315   22 /var       6104 -rw-r-----      82  w
root     syslogd     1315   23 /var       6099 -rw-r--r--  255549441  w
root     devd        1072 root /             3 drwxr-xr-x      32  r
root     devd        1072   wd /             3 drwxr-xr-x      32  r
root     devd        1072 text /          1124 -r-xr-xr-x  401160  r
root     devd        1072    0 /dev         31 crw-rw-rw-    null rw
root     devd        1072    1 /dev         31 crw-rw-rw-    null rw
root     devd        1072    2 /dev         31 crw-rw-rw-    null rw
root     devd        1072    3 /dev          5 crw-------  devctl  r
root     devd        1072    4* local stream ffffff0004ded000
root     devd        1072    5 /var         39 -rw-------       4  w
root     adjkerntz    545 root /             3 drwxr-xr-x      32  r
root     adjkerntz    545   wd /             3 drwxr-xr-x      32  r
root     adjkerntz    545 text /          1102 -r-xr-xr-x    9072  r
root     adjkerntz    545    0 /dev         31 crw-rw-rw-    null rw
root     adjkerntz    545    1 /dev         31 crw-rw-rw-    null rw
root     adjkerntz    545    2 /dev         31 crw-rw-rw-    null rw
root     zil_clean    455 root /             3 drwxr-xr-x      32  r
root     zil_clean    455   wd /             3 drwxr-xr-x      32  r
root     zil_clean    448 root /             3 drwxr-xr-x      32  r
root     zil_clean    448   wd /             3 drwxr-xr-x      32  r
root     zil_clean    435 root /             3 drwxr-xr-x      32  r
root     zil_clean    435   wd /             3 drwxr-xr-x      32  r
root     zil_clean    434 root /             3 drwxr-xr-x      32  r
root     zil_clean    434   wd /             3 drwxr-xr-x      32  r
root     zil_clean    433 root /             3 drwxr-xr-x      32  r
root     zil_clean    433   wd /             3 drwxr-xr-x      32  r
root     txg_thread_enter   423 root /             3 drwxr-xr-x      32  r
root     txg_thread_enter   423   wd /             3 drwxr-xr-x      32  r
root     txg_thread_enter   422 root /             3 drwxr-xr-x      32  r
root     txg_thread_enter   422   wd /             3 drwxr-xr-x      32  r
root     txg_thread_enter   421 root /             3 drwxr-xr-x      32  r
root     txg_thread_enter   421   wd /             3 drwxr-xr-x      32  r
root     vdev:worker ad14s1   420 root /             3 drwxr-xr-x      32  r
root     vdev:worker ad14s1   420   wd /             3 drwxr-xr-x      32  r
root     vdev:worker ad12s1   419 root /             3 drwxr-xr-x      32  r
root     vdev:worker ad12s1   419   wd /             3 drwxr-xr-x      32  r
root     vdev:worker ad6s1d   418 root /             3 drwxr-xr-x      32  r
root     vdev:worker ad6s1d   418   wd /             3 drwxr-xr-x      32  r
root     vdev:worker ad4s1d   417 root /             3 drwxr-xr-x      32  r
root     vdev:worker ad4s1d   417   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_5   416 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_5   416   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_5   415 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_5   415   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_5   414 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_5   414   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_5   413 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_5   413   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_5   412 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_5   412   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_5   411 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_5   411   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_5   410 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_5   410   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_5   409 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_5   409   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_4   408 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_4   408   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_4   407 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_4   407   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_4   406 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_4   406   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_4   405 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_4   405   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_4   404 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_4   404   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_4   403 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_4   403   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_4   402 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_4   402   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_4   401 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_4   401   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_3   400 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_3   400   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_3   399 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_3   399   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_3   398 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_3   398   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_3   397 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_3   397   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_3   396 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_3   396   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_3   395 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_3   395   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_3   394 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_3   394   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_3   393 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_3   393   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_2   392 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_2   392   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_2   391 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_2   391   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_2   390 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_2   390   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_2   389 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_2   389   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_2   388 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_2   388   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_2   387 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_2   387   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_2   386 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_2   386   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_2   385 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_2   385   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_1   384 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_1   384   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_1   383 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_1   383   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_1   382 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_1   382   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_1   381 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_1   381   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_1   380 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_1   380   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_1   379 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_1   379   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_1   378 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_1   378   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_1   377 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_1   377   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_0   376 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_0   376   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_0   375 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_0   375   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_0   374 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_0   374   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_0   373 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_0   373   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_0   372 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_0   372   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_0   371 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_0   371   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_0   370 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_0   370   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_0   369 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_0   369   wd /             3 drwxr-xr-x      32  r
root     txg_thread_enter   316 root /             3 drwxr-xr-x      32  r
root     txg_thread_enter   316   wd /             3 drwxr-xr-x      32  r
root     txg_thread_enter   315 root /             3 drwxr-xr-x      32  r
root     txg_thread_enter   315   wd /             3 drwxr-xr-x      32  r
root     txg_thread_enter   314 root /             3 drwxr-xr-x      32  r
root     txg_thread_enter   314   wd /             3 drwxr-xr-x      32  r
root     vdev:worker da0s2a   313 root /             3 drwxr-xr-x      32  r
root     vdev:worker da0s2a   313   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_5   312 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_5   312   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_5   311 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_5   311   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_5   310 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_5   310   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_5   309 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_5   309   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_5   308 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_5   308   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_5   307 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_5   307   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_5   306 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_5   306   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_5   305 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_5   305   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_4   304 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_4   304   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_4   303 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_4   303   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_4   302 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_4   302   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_4   301 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_4   301   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_4   300 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_4   300   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_4   299 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_4   299   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_4   298 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_4   298   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_4   297 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_4   297   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_3   296 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_3   296   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_3   295 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_3   295   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_3   294 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_3   294   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_3   293 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_3   293   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_3   292 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_3   292   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_3   291 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_3   291   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_3   290 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_3   290   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_3   289 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_3   289   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_2   288 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_2   288   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_2   287 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_2   287   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_2   286 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_2   286   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_2   285 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_2   285   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_2   284 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_2   284   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_2   283 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_2   283   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_2   282 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_2   282   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_2   281 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_2   281   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_1   280 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_1   280   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_1   279 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_1   279   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_1   278 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_1   278   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_1   277 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_1   277   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_1   276 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_1   276   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_1   275 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_1   275   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_1   274 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_1   274   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_1   273 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_1   273   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_0   272 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_0   272   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_0   271 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_0   271   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_0   270 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_0   270   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_0   269 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_0   269   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_0   268 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_0   268   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_0   267 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_0   267   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_0   266 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_0   266   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_0   265 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_0   265   wd /             3 drwxr-xr-x      32  r
root     USBPROC      196 root /             3 drwxr-xr-x      32  r
root     USBPROC      196   wd /             3 drwxr-xr-x      32  r
root     USBPROC      195 root /             3 drwxr-xr-x      32  r
root     USBPROC      195   wd /             3 drwxr-xr-x      32  r
root     USBPROC      194 root /             3 drwxr-xr-x      32  r
root     USBPROC      194   wd /             3 drwxr-xr-x      32  r
root     USBPROC      193 root /             3 drwxr-xr-x      32  r
root     USBPROC      193   wd /             3 drwxr-xr-x      32  r
root     USBPROC      192 root /             3 drwxr-xr-x      32  r
root     USBPROC      192   wd /             3 drwxr-xr-x      32  r
root     USBPROC      191 root /             3 drwxr-xr-x      32  r
root     USBPROC      191   wd /             3 drwxr-xr-x      32  r
root     USBPROC      172 root /             3 drwxr-xr-x      32  r
root     USBPROC      172   wd /             3 drwxr-xr-x      32  r
root     USBPROC      171 root /             3 drwxr-xr-x      32  r
root     USBPROC      171   wd /             3 drwxr-xr-x      32  r
root     zil_clean    145 root /             3 drwxr-xr-x      32  r
root     zil_clean    145   wd /             3 drwxr-xr-x      32  r
root     txg_thread_enter   144 root /             3 drwxr-xr-x      32  r
root     txg_thread_enter   144   wd /             3 drwxr-xr-x      32  r
root     txg_thread_enter   143 root /             3 drwxr-xr-x      32  r
root     txg_thread_enter   143   wd /             3 drwxr-xr-x      32  r
root     txg_thread_enter   142 root /             3 drwxr-xr-x      32  r
root     txg_thread_enter   142   wd /             3 drwxr-xr-x      32  r
root     vdev:worker ad14s2   141 root /             3 drwxr-xr-x      32  r
root     vdev:worker ad14s2   141   wd /             3 drwxr-xr-x      32  r
root     vdev:worker ad12s2   140 root /             3 drwxr-xr-x      32  r
root     vdev:worker ad12s2   140   wd /             3 drwxr-xr-x      32  r
root     USBPROC      139 root /             3 drwxr-xr-x      32  r
root     USBPROC      139   wd /             3 drwxr-xr-x      32  r
root     vdev:worker ad6s2   138 root /             3 drwxr-xr-x      32  r
root     vdev:worker ad6s2   138   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_5   137 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_5   137   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_5   136 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_5   136   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_5   135 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_5   135   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_5   134 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_5   134   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_5   133 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_5   133   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_5   132 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_5   132   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_5   131 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_5   131   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_5   130 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_5   130   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_4   129 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_4   129   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_4   128 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_4   128   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_4   127 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_4   127   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_4   126 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_4   126   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_4   125 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_4   125   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_4   124 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_4   124   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_4   123 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_4   123   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_4   122 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_4   122   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_3   121 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_3   121   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_3   120 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_3   120   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_3   119 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_3   119   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_3   118 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_3   118   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_3   117 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_3   117   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_3   116 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_3   116   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_3   115 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_3   115   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_3   114 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_3   114   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_2   113 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_2   113   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_2   112 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_2   112   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_2   111 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_2   111   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_2   110 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_2   110   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_2   109 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_2   109   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_2   108 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_2   108   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_2   107 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_2   107   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_2   106 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_2   106   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_1   105 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_1   105   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_1   104 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_1   104   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_1   103 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_1   103   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_1   102 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_1   102   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_1   101 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_1   101   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_1   100 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_1   100   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_1    99 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_1    99   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_1    98 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_1    98   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_0    97 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_0    97   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_0    96 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_0    96   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_0    95 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_0    95   wd /             3 drwxr-xr-x      32  r
root     spa_zio_intr_0    94 root /             3 drwxr-xr-x      32  r
root     spa_zio_intr_0    94   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_0    93 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_0    93   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_0    92 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_0    92   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_0    91 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_0    91   wd /             3 drwxr-xr-x      32  r
root     spa_zio_issue_0    90 root /             3 drwxr-xr-x      32  r
root     spa_zio_issue_0    90   wd /             3 drwxr-xr-x      32  r
root     USBPROC       89 root /             3 drwxr-xr-x      32  r
root     USBPROC       89   wd /             3 drwxr-xr-x      32  r
root     init           1 root /             3 drwxr-xr-x      32  r
root     init           1   wd /             3 drwxr-xr-x      32  r
root     init           1 text /          1197 -r-xr-xr-x  705696  r
root     kernel         0 root /             3 drwxr-xr-x      32  r
root     kernel         0   wd /             3 drwxr-xr-x      32  r

------------------------------------------------------------------------
dmesg

Copyright (c) 1992-2008 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 8.0-CURRENT #1: Wed Oct 15 21:39:04 BST 2008
    sandbox at w2fzz0vc03.aah-go-on.com:/usr/home/sandbox/newusb/obj/usr/home/sandbox/newusb/src/sys/GENERIC_usb2
WARNING: WITNESS option enabled, expect reduced performance.
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Core(TM)2 Quad CPU           @ 2.66GHz (2660.04-MHz K8-class CPU)
  Origin = "GenuineIntel"  Id = 0x6f7  Stepping = 7
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0xe3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM>
  AMD Features=0x20100800<SYSCALL,NX,LM>
  AMD Features2=0x1<LAHF>
  Cores per package: 4
usable memory = 4276260864 (4078 MB)
avail memory  = 4091035648 (3901 MB)
ACPI APIC Table: <DELL   B8K    >
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  1
 cpu2 (AP): APIC ID:  2
 cpu3 (AP): APIC ID:  3
This module (opensolaris) contains code covered by the
Common Development and Distribution License (CDDL)
see http://opensolaris.org/os/licensing/opensolaris_license/
ioapic0: Changing APIC ID to 8
ioapic0 <Version 1.1> irqs 0-23 on motherboard
lapic0: Forcing LINT1 to edge trigger
kbd1 at kbdmux0
ath_hal: 0.10.5.10 (AR5210, AR5211, AR5212, AR5416, RF5111, RF5112, RF2413, RF5413, RF2133, RF2425, RF2417)
acpi0: <DELL B8K    > on motherboard
acpi0: [ITHREAD]
acpi0: Power Button (fixed)
acpi0: reservation of 0, a0000 (3) failed
acpi0: reservation of 100000, f00000 (3) failed
acpi0: reservation of 1000000, 9edbcc00 (3) failed
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x4008-0x400b on acpi0
acpi_hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 25000000 Hz quality 900
acpi_button0: <Power Button> on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pci0: <memory, RAM> at device 0.1 (no driver attached)
pci0: <memory, RAM> at device 0.2 (no driver attached)
pci0: <memory, RAM> at device 0.3 (no driver attached)
pci0: <memory, RAM> at device 0.4 (no driver attached)
pci0: <memory, RAM> at device 1.0 (no driver attached)
pci0: <memory, RAM> at device 1.1 (no driver attached)
pci0: <memory, RAM> at device 1.2 (no driver attached)
pci0: <memory, RAM> at device 1.3 (no driver attached)
pci0: <memory, RAM> at device 1.4 (no driver attached)
pci0: <memory, RAM> at device 1.5 (no driver attached)
pci0: <memory, RAM> at device 1.6 (no driver attached)
pcib1: <ACPI PCI-PCI bridge> at device 2.0 on pci0
pci1: <ACPI PCI bus> on pcib1
vgapci0: <VGA-compatible display> port 0xdc80-0xdcff mem 0xde000000-0xdeffffff,0xa0000000-0xafffffff,0xdc000000-0xddffffff irq 16 at device 0.0 on pci1
pcib2: <ACPI PCI-PCI bridge> at device 4.0 on pci0
pci2: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> at device 5.0 on pci0
pci3: <ACPI PCI bus> on pcib3
pci0: <memory, RAM> at device 9.0 (no driver attached)
isab0: <PCI-ISA bridge> port 0x4f00-0x4fff at device 10.0 on pci0
isa0: <ISA bus> on isab0
pci0: <serial bus, SMBus> at device 10.1 (no driver attached)
ohci0: <OHCI (generic) USB controller> mem 0xdfffc000-0xdfffcfff irq 21 at device 11.0 on pci0
ohci0: [ITHREAD]
usbus0: <OHCI (generic) USB controller> on ohci0
ehci0: <EHCI (generic) USB 2.0 controller> mem 0xdfffbf00-0xdfffbfff irq 22 at device 11.1 on pci0
ehci0: [ITHREAD]
usbus1: waiting for BIOS to give up control
usbus1: EHCI version 1.0
usbus1: <EHCI (generic) USB 2.0 controller> on ehci0
atapci0: <nVidia nForce MCP55 UDMA133 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xecf0-0xecff at device 13.0 on pci0
ata0: <ATA channel 0> on atapci0
ata0: [ITHREAD]
ata1: <ATA channel 1> on atapci0
ata1: [ITHREAD]
atapci1: <nVidia nForce MCP55 SATA300 controller> port 0xfe00-0xfe07,0xfe10-0xfe13,0xfe20-0xfe27,0xfe30-0xfe33,0xfec0-0xfecf mem 0xdfffd000-0xdfffdfff irq 23 at device 14.0 on pci0
atapci1: [ITHREAD]
ata2: <ATA channel 0> on atapci1
ata2: [ITHREAD]
ata3: <ATA channel 1> on atapci1
ata3: [ITHREAD]
atapci2: <nVidia nForce MCP55 SATA300 controller> port 0xfe40-0xfe47,0xfe50-0xfe53,0xfe60-0xfe67,0xfe70-0xfe73,0xfed0-0xfedf mem 0xdfffe000-0xdfffefff irq 20 at device 14.1 on pci0
atapci2: [ITHREAD]
ata4: <ATA channel 0> on atapci2
ata4: [ITHREAD]
ata5: <ATA channel 1> on atapci2
ata5: [ITHREAD]
atapci3: <nVidia nForce MCP55 SATA300 controller> port 0xfe80-0xfe87,0xfe90-0xfe93,0xfea0-0xfea7,0xfeb0-0xfeb3,0xfef0-0xfeff mem 0xdffff000-0xdfffffff irq 21 at device 14.2 on pci0
atapci3: [ITHREAD]
ata6: <ATA channel 0> on atapci3
ata6: [ITHREAD]
ata7: <ATA channel 1> on atapci3
ata7: [ITHREAD]
pcib4: <ACPI PCI-PCI bridge> at device 15.0 on pci0
pci4: <ACPI PCI bus> on pcib4
pci4: <unknown> at device 4.0 (no driver attached)
pci4: <multimedia, audio> at device 5.0 (no driver attached)
fwohci0: <Texas Instruments TSB43AB22/A> mem 0xdadfb800-0xdadfbfff,0xdadfc000-0xdadfffff irq 18 at device 10.0 on pci4
fwohci0: [FILTER]
fwohci0: OHCI version 1.10 (ROM=0)
fwohci0: No. of Isochronous channels is 4.
fwohci0: EUI64 80:00:00:00:00:00:00:00
fwohci0: Phy 1394a available S400, 2 ports.
fwohci0: Link S400, max_rec 2048 bytes.
firewire0: <IEEE1394(FireWire) bus> on fwohci0
dcons_crom0: <dcons configuration ROM> on firewire0
dcons_crom0: bus_addr 0x2598000
fwe0: <Ethernet over FireWire> on firewire0
if_fwe0: Fake Ethernet address: 82:00:00:00:00:00
fwe0: Ethernet address: 82:00:00:00:00:00
fwip0: <IP over FireWire> on firewire0
fwip0: Firewire address: 80:00:00:00:00:00:00:00 @ 0xfffe00000000, S400, maxrec 2048
sbp0: <SBP-2/SCSI over FireWire> on firewire0
fwohci0: Initiate bus reset
fwohci0: BUS reset
fwohci0: node_id=0xc800ffc0, gen=1, CYCLEMASTER mode
pcib5: <ACPI PCI-PCI bridge> at device 19.0 on pci0
pci5: <ACPI PCI bus> on pcib5
pcib6: <ACPI PCI-PCI bridge> at device 24.0 on pci0
pci6: <ACPI PCI bus> on pcib6
vgapci1: <VGA-compatible display> port 0xbc80-0xbcff mem 0xd1000000-0xd1ffffff,0xc0000000-0xcfffffff,0xd2000000-0xd3ffffff irq 16 at device 0.0 on pci6
atrtc0: <AT realtime clock> port 0x70-0x7f irq 8 on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
atkbd0: [ITHREAD]
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: [ITHREAD]
psm0: model MouseMan+, device ID 0
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart0: [FILTER]
cpu0: <ACPI CPU> on acpi0
est0: <Enhanced SpeedStep Frequency Control> on cpu0
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr a2a0a2a86000a2a
device_attach: est0 attach returned 6
p4tcc0: <CPU Frequency Thermal Control> on cpu0
cpu1: <ACPI CPU> on acpi0
est1: <Enhanced SpeedStep Frequency Control> on cpu1
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr a2a0a2a86000a2a
device_attach: est1 attach returned 6
p4tcc1: <CPU Frequency Thermal Control> on cpu1
cpu2: <ACPI CPU> on acpi0
est2: <Enhanced SpeedStep Frequency Control> on cpu2
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr a2a0a2a86000a2a
device_attach: est2 attach returned 6
p4tcc2: <CPU Frequency Thermal Control> on cpu2
cpu3: <ACPI CPU> on acpi0
est3: <Enhanced SpeedStep Frequency Control> on cpu3
est: CPU supports Enhanced Speedstep, but is not recognized.
est: cpu_vendor GenuineIntel, msr a2a0a2a86000a2a
device_attach: est3 attach returned 6
p4tcc3: <CPU Frequency Thermal Control> on cpu3
orm0: <ISA Option ROMs> at iomem 0xc0000-0xcc7ff,0xcc800-0xce7ff,0xce800-0xcffff on isa0
ppc0: cannot reserve I/O port range
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
WARNING: ZFS is considered to be an experimental feature in FreeBSD.
Timecounters tick every 1.000 msec
usbus0: 12Mbps Full Speed USB v1.0
firewire0: 1 nodes, maxhop <= 0, cable IRM = 0 (me)
firewire0: bus manager 0 (me)
ZFS filesystem version 6
ZFS storage pool version 6
ugen0.1: <nVidia> at usbus0
ushub0: <nVidia OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
ushub0: 10 ports with 10 removable, self powered
usbus1: 480Mbps High Speed USB v2.0
ugen1.1: <nVidia> at usbus1
ushub1: <nVidia EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
ushub1: 10 ports with 10 removable, self powered
ad4: 305245MB <Hitachi HDP725032GLA360 GM3OA52A> at ata2-master SATA300
ad6: 305245MB <Hitachi HDP725032GLA360 GM3OA52A> at ata3-master SATA300
acd0: CDRW <TSSTcorp CD-RW/DVD-ROM TS-H493A/D200> at ata4-master SATA150
acd1: DVDR <TSSTcorp DVD+/-RW TS-H653A/D500> at ata5-master SATA150
GEOM_LABEL: Label for provider ad4s2 is ntfs/System.
ad12: 305245MB <Hitachi HDT725032VLA360 V54OA7EA> at ata6-master SATA300
ad14: 305245MB <Hitachi HDT725032VLA360 V54OA7EA> at ata7-master SATA300
lapic1: Forcing LINT1 to edge trigger
SMP: AP CPU #1 Launched!
lapic2: Forcing LINT1 to edge trigger
SMP: AP CPU #2 Launched!
lapic3: Forcing LINT1 to edge trigger
SMP: AP CPU #3 Launched!
WARNING: WITNESS option enabled, expect reduced performance.
Trying to mount root from zfs:tank
lock order reversal:
 1st 0xffffff0002d72000 vfslock (vfslock) @ /usr/home/sandbox/newusb/src/sys/kern/vfs_subr.c:372
 2nd 0xffffff0002d73448 devfs (devfs) @ /usr/home/sandbox/newusb/src/sys/kern/vfs_lookup.c:428
 3rd 0xffffff0002d71c80 vfslock (vfslock) @ /usr/home/sandbox/newusb/src/sys/kern/vfs_subr.c:372
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
_witness_debugger() at _witness_debugger+0x49
witness_checkorder() at witness_checkorder+0x7a5
__lockmgr_args() at __lockmgr_args+0xca6
vfs_busy() at vfs_busy+0xd0
vfs_mount_alloc() at vfs_mount_alloc+0x7d
vfs_donmount() at vfs_donmount+0xf05
kernel_mount() at kernel_mount+0xa1
kernel_vmount() at kernel_vmount+0xce
vfs_mountroot_try() at vfs_mountroot_try+0xf4
vfs_mountroot() at vfs_mountroot+0x3ea
start_init() at start_init+0x62
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xfffffffe40023d40, rbp = 0 ---
ugen1.2: <Seagate> at usbus1
umass0: <Interface0> on usbus1
umass0:  SCSI over Bulk-Only; quirks = 0x0000
lock order reversal:
 1st 0xffffff0002c4a7f8 zfs (zfs) @ /usr/home/sandbox/newusb/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:1123
 2nd 0xffffff0002d72000 vfslock (vfslock) @ /usr/home/sandbox/newusb/src/sys/kern/vfs_subr.c:372
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
_witness_debugger() at _witness_debugger+0x49
witness_checkorder() at witness_checkorder+0x7a5
__lockmgr_args() at __lockmgr_args+0xca6
vfs_busy() at vfs_busy+0xd0
lookup() at lookup+0x8c0
namei() at namei+0x454
kern_unlinkat() at kern_unlinkat+0x9d
vfs_mountroot_try() at vfs_mountroot_try+0x411
vfs_mountroot() at vfs_mountroot+0x3ea
start_init() at start_init+0x62
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xfffffffe40023d40, rbp = 0 ---
lock order reversal:
 1st 0xffffff00024ee070 user map (user map) @ /usr/home/sandbox/newusb/src/sys/vm/vm_map.c:3115
 2nd 0xffffff0002c4a448 zfs (zfs) @ /usr/home/sandbox/newusb/src/sys/kern/vfs_subr.c:2049
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
_witness_debugger() at _witness_debugger+0x49
witness_checkorder() at witness_checkorder+0x7a5
__lockmgr_args() at __lockmgr_args+0xca6
vop_stdlock() at vop_stdlock+0x39
VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b
_vn_lock() at umass0:1:0:-1: At_tvanc_hleodc kt+o0 xs5c0bu
s1
vget() at vget+0x7d
vnode_pager_lock() at vnode_pager_lock+0x1d4
vm_fault() at vm_fault+0x22f
trap_pfault() at trap_pfault+0x103
trap() at trap+0x51c
calltrap() at calltrap+0x8
--- trap 0xc, rip = 0x40014f, rsp = 0x7fffffffee70, rbp = 0x7fffffffee90 ---
da0 at umass-sim0 bus 0 target 0 lun 0
da0: <Seagate FreeAgentDesktop 100D> Fixed Direct Access SCSI-4 device 
da0: 40.000MB/s transfers
da0: 476940MB (976773168 512 byte sectors: 255H 63S/T 60801C)
GEOM_LABEL: Label for provider da0s1 is ntfs/Backup.
Entropy harvesting:
 interrupts
 ethernet
 point_to_point
ugen1.3: <Belkin Components> at usbus1
ushub2: <Belkin Components F5U237 USB 2.0 7-Port Hub, class 9/0, rev 2.00/1.00, addr 3> on usbus1
 kickstart
.
ushub2: 7 ports with 7 removable, self powered
ugen1.4: <MARVELL .......> at usbus1
ugen1.5: <Belkin> at usbus1
rum0: <Belkin Belkin 54g USB Network Adapter, class 0/0, rev 2.00/0.01, addr 5> on usbus1
rum0: MAC/BBP RT2573 (rev 0x2573a), RF RT2528
ugen1.6: <TEAC> at usbus1
umass1: <TEAC CA-200, class 0/0, rev 2.00/4.00, addr 6> on usbus1
umass1:  SCSI over Bulk-Only; quirks = 0x0000
umass1:2:1:-1: Attached to scbus2
da1 at umass-sim1 bus 1 target 0 lun 0
da1: <TEAC USB   HS-CF Card 4.00> Removable Direct Access SCSI-0 device 
da1: 40.000MB/s transfers
da1: Attempt to query device size failed: NOT READY, Medium not present
da2 at umass-sim1 bus 1 target 0 lun 1
da2: <TEAC USB   HS-xD/SM 4.00> Removable Direct Access SCSI-0 device 
da2: 40.000MB/s transfers
da2: Attempt to query device size failed: NOT READY, Medium not present
da3 at umass-sim1 bus 1 target 0 lun 2
da3: <TEAC USB   HS-MS Card 4.00> Removable Direct Access SCSI-0 device 
da3: 40.000MB/s transfers
da3: Attempt to query device size failed: NOT READY, Medium not present
da4 at umass-sim1 bus 1 target 0 lun 3
da4: <TEAC USB   HS-SD Card 4.00> Removable Direct Access SCSI-0 device 
da4: 40.000MB/s transfers
da4: Attempt to query device size failed: NOT READY, Medium not present
/dev/ad4s1a: 6872 files, 492217 used, 520797 free (3413 frags, 64673 blocks, 0.3% fragmentation)
Mismatched versions:  File system is version 2 on-disk format, which is incompatible with this software version 1!
cannot mount 'backup/20080823-0125': Operation not supported
lock order reversal:
 1st 0xffffff0004b7d098 syncer (syncer) @ /usr/home/sandbox/newusb/src/sys/kern/vfs_subr.c:1657
 2nd 0xffffff0004ba7960 vfslock (vfslock) @ /usr/home/sandbox/newusb/src/sys/kern/vfs_subr.c:372
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
_witness_debugger() at _witness_debugger+0x49
witness_checkorder() at witness_checkorder+0x7a5
__lockmgr_args() at __lockmgr_args+0xca6
vfs_busy() at vfs_busy+0xd0
sync_fsync() at sync_fsync+0x9c
sync_vnode() at sync_vnode+0x143
sched_sync() at sched_sync+0x1c5
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xfffffffeb4d05d40, rbp = 0 ---
wlan0: Ethernet address: 00:17:3f:72:40:90
Starting Network: lo0.
add net default: gateway 192.168.0.1
Additional TCP/IP options:
 tcp extensions=NO
.
savecore: reboot after panic: mutex Giant not owned at /usr/home/sandbox/newusb/src/sys/kern/tty_ttydisc.c:1129
Oct 16 15:27:09 w2fzz0vc03 savecore: reboot after panic: mutex Giant not owned at /usr/home/sandbox/newusb/src/sys/kern/tty_ttydisc.c:1129
savecore: writing core to vmcore.41
wlan0: link state changed to UP
Additional ABI support:
 linux
.
Setting date via ntp.
16 Oct 15:28:06 ntpdate[1341]: step time server 192.108.114.23 offset 0.827552 sec
Setting NIS domain: aah-go-on.com.
pg_ctl: invalid data in PID file "/usr/local/pgsql/data/postmaster.pid"
Oct 16 15:28:08 w2fzz0vc03 mDNSResponder (Engineering Build) (Aug 19 2008 18:02:49) [1558]: starting
Configuring syscons:
 keymap
 font8x16
 font8x14
 font8x8
 blanktime
 screensaver
.
Local package initialization:
 uptimed
.

Thu Oct 16 15:28:11 BST 2008
Oct 16 15:31:28 w2fzz0vc03 login: ROOT LOGIN (root) ON ttyv0
Oct 16 15:41:04 w2fzz0vc03 login: ROOT LOGIN (root) ON ttyv1
Kernel page fault with the following non-sleepable locks held:
exclusive sleep mutex UMASS lock (UMASS lock) r = 0 (0xffffffff80b67e60) locked @ /usr/home/sandbox/newusb/src/sys/dev/usb2/core/usb2_transfer.c:1795
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
_witness_debugger() at _witness_debugger+0x49
witness_warn() at witness_warn+0x232
trap() at trap+0x38f
calltrap() at calltrap+0x8
--- trap 0xc, rip = 0xffffffff807b2a46, rsp = 0xfffffffeb7267ac0, rbp = 0xfffffffeb7267b00 ---
bcopy() at bcopy+0x16
usb2_bdma_pre_sync() at usb2_bdma_pre_sync+0x59
usb2_bdma_work_loop() at usb2_bdma_work_loop+0x2ac
usb2_command_wrapper() at usb2_command_wrapper+0x76
usb2_callback_wrapper() at usb2_callback_wrapper+0x101
usb2_command_wrapper() at usb2_command_wrapper+0x76
usb2_callback_proc() at usb2_callback_proc+0x68
usb2_process() at usb2_process+0xc0
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xfffffffeb7267d40, rbp = 0 ---


Fatal trap 12: page fault while in kernel mode
cpuid = 3; apic id = 03
fault virtual address	= 0xfffffffe69a00000
fault code		= supervisor read data, page not present
instruction pointer	= 0x8:0xffffffff807b2a46
stack pointer	        = 0x10:0xfffffffeb7267ac0
frame pointer	        = 0x10:0xfffffffeb7267b00
code segment		= base 0x0, limit 0xfffff, type 0x1b
			= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags	= interrupt enabled, resume, IOPL = 0
current process		= 139 (USBPROC)
lock order reversal: (Giant after non-sleepable)
 1st 0xffffffff80b67e60 UMASS lock (UMASS lock) @ /usr/home/sandbox/newusb/src/sys/dev/usb2/core/usb2_transfer.c:1795
 2nd 0xffffffff80b6bfe0 Giant (Giant) @ /usr/home/sandbox/newusb/src/sys/dev/kbdmux/kbdmux.c:1044
KDB: stack backtrace:
panic: mutex Giant not owned at /usr/home/sandbox/newusb/src/sys/kern/tty_ttydisc.c:1129
cpuid = 3
KDB: stack backtrace:
Physical memory: 4078 MB
Dumping 1154 MB: 1139 1123 1107 1091 1075 1059 1043 1027 1011 995 979 963 947 931 915 899 883 867 851 835 819 803 787 771 755 739 723 707 691 675 659 643 627 611 595 579 563 547 531 515 499 483 467 451 435 419 403 387 371 355 339 323 307 291 275 259 243 227 211 195 179 163 147 131 115 99 83 67 51 35 19 3

------------------------------------------------------------------------
kernel config

config: File /boot/kernel/kernel doesn't contain configuration file. Either unsupported, or not compiled with INCLUDE_CONFIG_FILE
 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: core.txt.42
Type: text/x-diff
Size: 127988 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-current/attachments/20081107/6a0b1af0/core.txt-0001.bin


More information about the freebsd-current mailing list