kernel panic ZFS related on 10.3-RELEASE

Wed May 11 19:26:45 UTC 2016

Hello,

It's been a while I didn't have a kernel panic, it just happened today
on my 10.3 machine. It looks like being ZFS related.

GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and
you are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for
details.
This GDB was configured as "amd64-marcel-freebsd"...

Unread portion of the kernel message buffer:

Fatal trap 9: general protection fault while in kernel mode
cpuid = 0; apic id = 00
instruction pointer	= 0x20:0xffffffff81a0898c
stack pointer	        = 0x28:0xfffffe0463cf7690
frame pointer	        = 0x28:0xfffffe0463cf76e0
code segment		= base 0x0, limit 0xfffff, type 0x1b
			= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags	= interrupt enabled, resume, IOPL = 0
current process		= 18 (syncer)
trap number		= 9
panic: general protection fault
cpuid = 0
KDB: stack backtrace:
#0 0xffffffff8099f3b0 at kdb_backtrace+0x60
#1 0xffffffff80962086 at vpanic+0x126
#2 0xffffffff80961f53 at panic+0x43
#3 0xffffffff80d87d8b at trap_fatal+0x36b
#4 0xffffffff80d87a0d at trap+0x77d
#5 0xffffffff80d6d6e2 at calltrap+0x8
#6 0xffffffff81a08dd8 at avl_remove+0x1c8
#7 0xffffffff81ac7eae at zfs_range_unlock+0xce
#8 0xffffffff81acb5f7 at zfs_get_done+0x37
#9 0xffffffff81acb8b3 at zfs_get_data+0x263
#10 0xffffffff81aa1d9c at zil_commit+0x67c
#11 0xffffffff81acb008 at zfs_sync+0xa8
#12 0xffffffff80a0c446 at sync_fsync+0x136
#13 0xffffffff80eb5327 at VOP_FSYNC_APV+0xa7
#14 0xffffffff80a0ce3b at sched_sync+0x3ab
#15 0xffffffff8092b50a at fork_exit+0x9a
#16 0xffffffff80d6dc1e at fork_trampoline+0xe
Uptime: 20d0h20m26s
Dumping 3609 out of 16255
MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%

Reading symbols from /boot/kernel/zfs.ko.symbols...done.
Loaded symbols for /boot/kernel/zfs.ko.symbols
Reading symbols from /boot/kernel/opensolaris.ko.symbols...done.
Loaded symbols for /boot/kernel/opensolaris.ko.symbols
Reading symbols from /boot/kernel/pflog.ko.symbols...done.
Loaded symbols for /boot/kernel/pflog.ko.symbols
Reading symbols from /boot/kernel/pf.ko.symbols...done.
Loaded symbols for /boot/kernel/pf.ko.symbols
Reading symbols from /boot/kernel/linux.ko.symbols...done.
Loaded symbols for /boot/kernel/linux.ko.symbols
Reading symbols from /boot/kernel/linux_common.ko.symbols...done.
Loaded symbols for /boot/kernel/linux_common.ko.symbols
Reading symbols from /boot/kernel/linux64.ko.symbols...done.
Loaded symbols for /boot/kernel/linux64.ko.symbols
Reading symbols from /boot/kernel/nullfs.ko.symbols...done.
Loaded symbols for /boot/kernel/nullfs.ko.symbols
Reading symbols from /boot/kernel/linprocfs.ko.symbols...done.
Loaded symbols for /boot/kernel/linprocfs.ko.symbols
Reading symbols from /boot/kernel/fdescfs.ko.symbols...done.
Loaded symbols for /boot/kernel/fdescfs.ko.symbols
Reading symbols from /boot/kernel/tmpfs.ko.symbols...done.
Loaded symbols for /boot/kernel/tmpfs.ko.symbols
Reading symbols from /boot/kernel/linsysfs.ko.symbols...done.
Loaded symbols for /boot/kernel/linsysfs.ko.symbols
#0  doadump (textdump=<value optimized out>) at pcpu.h:219
219		__asm("movq %%gs:%1,%0" : "=r" (td)
(kgdb) list *0xffffffff81a0898c
0xffffffff81a0898c is in avl_rotation
(/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/common/avl/avl
.c:316).
311		avl_node_t *cright;
312		avl_node_t *gchild;
313		avl_node_t *gright;
314		avl_node_t *gleft;
315		int which_child = AVL_XCHILD(node);
316		int child_bal = AVL_XBALANCE(child);
317	
318		/* BEGIN CSTYLED */
319		/*
320		 * case 1 : node is overly left heavy, the left
child is balanced or
Current language:  auto; currently minimal
(kgdb) bt
#0  doadump (textdump=<value optimized out>) at pcpu.h:219
#1  0xffffffff80961ce2 in kern_reboot (howto=260) at
/usr/src/sys/kern/kern_shutdown.c:486
#2  0xffffffff809620c5 in vpanic (fmt=<value optimized out>, ap=<value
optimized out>) at /usr/src/sys/kern/kern_shutdown.c:889
#3  0xffffffff80961f53 in panic (fmt=0x0) at
/usr/src/sys/kern/kern_shutdown.c:818
#4  0xffffffff80d87d8b in trap_fatal (frame=<value optimized out>,
eva=<value optimized out>) at /usr/src/sys/amd64/amd64/trap.c:858
#5  0xffffffff80d87a0d in trap (frame=<value optimized out>) at
/usr/src/sys/amd64/amd64/trap.c:203
#6  0xffffffff80d6d6e2 in calltrap () at
/usr/src/sys/amd64/amd64/exception.S:236
#7  0xffffffff81a0898c in avl_rotation (tree=0xfffff801ab4e2d90,
node=0xffffffff81d05aa8, balance=<value optimized out>)
    at
/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/common/avl/avl.
c:315
#8  0xffffffff81a08dd8 in avl_remove (tree=0xfffff801ab4e2d90,
data=<value optimized out>)
    at
/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/common/avl/avl.
c:818
#9  0xffffffff81ac7eae in zfs_range_unlock (rl=0xfffff8025c748900) at
/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/z
fs/zfs_rlock.c:477
#10 0xffffffff81acb5f7 in zfs_get_done (zgd=0xfffff8006c6f7900,
error=2) at
/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/z
fs/zfs_vnops.c:1224
#11 0xffffffff81acb8b3 in zfs_get_data (arg=<value optimized out>,
lr=<value optimized out>, buf=<value optimized out>, zio=<value
optimized out>)
    at
/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/z
fs/zfs_vnops.c:1363
#12 0xffffffff81aa1d9c in zil_commit (zilog=0xfffff801006fb000,
foid=<value optimized out>)
    at
/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/z
fs/zil.c:1110
#13 0xffffffff81acb008 in zfs_sync (vfsp=<value optimized out>,
waitfor=<value optimized out>)
    at
/usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/z
fs/zfs_vfsops.c:161
#14 0xffffffff80a0c446 in sync_fsync (ap=<value optimized out>) at
/usr/src/sys/kern/vfs_subr.c:3857
#15 0xffffffff80eb5327 in VOP_FSYNC_APV (vop=<value optimized out>,
a=<value optimized out>) at vnode_if.c:1330
#16 0xffffffff80a0ce3b in sched_sync () at vnode_if.h:549
#17 0xffffffff8092b50a in fork_exit (callout=0xffffffff80a0ca90
<sched_sync>, arg=0x0, frame=0xfffffe0463cf7ac0) at
/usr/src/sys/kern/kern_fork.c:1027
#18 0xffffffff80d6dc1e in fork_trampoline () at
/usr/src/sys/amd64/amd64/exception.S:611
#19 0x0000000000000000 in ?? ()

What can I do to provide more info?

Regards,

-- 
David Demelier