ZFS panic: solaris assert: arc_buf_remove_buf(db->db_buf, db) == 0

[Peter Schuller]

Hello,

I am experiencing a panic that I do not believe to have been discussed
already. It is being triggered while rsync:ing a directory hierarchy
from another machine over ssh onto a 6 drive raidz2 pool. The transfer
consists mostly of large files (rather than a huge amount of small
files). I seem to be getting this panic within a few hours of starting
or resuming said transfer.

The machine only has 512+256 minus some megs stolen by built-in VGA of
memory.

The following is manually transcribed because crashdumps do not work[1],
so there may be typos:

panic: solaris assert: arc_buf_remove_buf(db->db_buf, db) == 0,file:
/usr/src/sys/modules/zfs/../../contrib/opensolaris/uts/common/fs/zfs/dbuf.c,
line: 1718
cpuid = 0
KBD: enter: panic
[thread pid 132 tid 100072]
Stopped at kbd_enter+0x2b: nop
db> bt
Tracing pid 132 tid 100072 td 0xc3899d80
kbd_enter(c095f2c7) at kbd_enter+0x2b
panic(c0c5ab69,c0c5b020,c0c5af8c,6b6,a99a,...) at pnic+0x11c
dbuf_rele(cb874578,a99a,cb874578,a99a,0,...) at dbuf_rele+0x18c
arc_write_done(c940b000,c940b200,c39aa000,c7663000,cbe93000,...) at
arc_write_done+0x13a
zio_done(c940b000,c0c3380f,c45fec00,0,1,...) at zio_done+0x1ee
zio_vdev_io_assess(c940b000,c37bd104,c37bd124,c37bd148,929a53f1,...) at
zio_vdev_io_assess+0x116
taskq_thread(c37bd0cc,ddb85d38) at taskq_thread+0x101
fork_exit(c0bfa2c4,c37bd0cc,ddb85d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xddb85db70, ebp = 0 ---
db>

dmesg for the machine:

http://distfiles.scode.org/mlref/freebsd-dmesg-zfs-panic-arcbufremove-7current.txt

It's a version of current compiled somewhere leading up to May 8.

Contents of /boot/loader.conf:

linux_load="YES"
geom_label_load="YES"
zfs_load="YES"
vm.kmem_size=272629760

Contents of /etc/sysctl.conf:

kern.ps_arg_cache_limit=4096
kern.maxvnodes=15000

I have gotten this panic twice confirmed so far. The stacktrace is from
the second instance. I did not grab the trace the first time because I
was hoping for the crashdump.

[1] Even after manually running "dumpon" and making sure it suceeded.
Are crashdumps supposed to work on glabel devices? That is the only
"special" circumstance I am aware of.


-- 
/ Peter Schuller

PGP userID: 0xE9758B7D or 'Peter Schuller <peter.schuller at infidyne.com>'
Key retrieval: Send an E-Mail to getpgpkey at scode.org
E-Mail: peter.schuller at infidyne.com Web: http://www.scode.org


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 250 bytes
Desc: OpenPGP digital signature
Url : http://lists.freebsd.org/pipermail/freebsd-fs/attachments/20070517/53f2b19b/signature.pgp