Crashes (with dumps) in zap_leaf.c

[Zaphod Beeblebrox]

Replying to my own post and adding some detail.  I didn't realize at first
that was an l as-in the lowercase L.  But looking at the macro, it is.

(kgdb) p l
$1 = (zap_leaf_t *) 0xfffff8008b3a3b00
(kgdb) p *l
$2 = {l_dbu = {dbu_tqent = {tqent_task = {ta_link = {stqe_next = 0x0},
        ta_pending = 0, ta_priority = 0, ta_func = 0, ta_context = 0x0},
      tqent_func = 0, tqent_arg = 0x0},
    dbu_evict_func = 0xffffffff81a482b0 <zap_leaf_pageout>}, l_rwlock = {
    lock_object = {lo_name = 0xffffffff81af06d9 "l->l_rwlock",
      lo_flags = 40960000, lo_data = 0, lo_witness = 0x0},
    sx_lock = 18446735297203916800}, l_blkid = 1311, l_bs = 14,
  l_dbuf = 0xfffff806255e11b0}

Now the macro starts by dereferencing l->l_phys->l_hash ... I can get it to
spit out l->l_bs, but the debugger won't spit out l->l_phys.

At this point ... I'm wondering if I'm confused and missing something.


On Tue, Aug 2, 2016 at 1:37 AM, Zaphod Beeblebrox <zbeeble at gmail.com> wrote:

> I'm getting multiple crashes that look rather like:
>
> #0 0xffffffff8098e390 at kdb_backtrace+0x60
> #1 0xffffffff80951066 at vpanic+0x126
> #2 0xffffffff80950f33 at panic+0x43
> #3 0xffffffff80bcfa4c at vm_fault_hold+0x1b2c
> #4 0xffffffff80bcded7 at vm_fault+0x77
> #5 0xffffffff80d5612c at trap_pfault+0x19c
> #6 0xffffffff80d558fa at trap+0x47a
> #7 0xffffffff80d3b8d2 at calltrap+0x8
> #8 0xffffffff81a49a5a at zap_entry_create+0x27a
> #9 0xffffffff81a45eee at fzap_add_cd+0xde
> #10 0xffffffff81a4c051 at zap_add+0x101
> #11 0xffffffff81a6bfb5 at zfs_link_create+0x415
> #12 0xffffffff81a87a34 at zfs_freebsd_rename+0xac4
> #13 0xffffffff80e81e1b at VOP_RENAME_APV+0xab
> #14 0xffffffff80a03476 at kern_renameat+0x4a6
> #15 0xffffffff80d5694f at amd64_syscall+0x40f
> #16 0xffffffff80d3bbbb at Xfast_syscall+0xfb
>
> Analyzing one of those crashes:
>
> #0  doadump (textdump=<value optimized out>) at pcpu.h:219
> 219     pcpu.h: No such file or directory.
>         in pcpu.h
> (kgdb) bt
> #0  doadump (textdump=<value optimized out>) at pcpu.h:219
> #1  0xffffffff80950cc2 in kern_reboot (howto=260)
>     at /usr/src/sys/kern/kern_shutdown.c:486
> #2  0xffffffff809510a5 in vpanic (fmt=<value optimized out>,
>     ap=<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:889
> #3  0xffffffff80950f33 in panic (fmt=0x0)
>     at /usr/src/sys/kern/kern_shutdown.c:818
> #4  0xffffffff80bcfa4c in vm_fault_hold (map=<value optimized out>,
>     vaddr=<value optimized out>, fault_type=<value optimized out>,
>     fault_flags=<value optimized out>, m_hold=<value optimized out>)
>     at /usr/src/sys/vm/vm_fault.c:329
> #5  0xffffffff80bcded7 in vm_fault (map=0xfffff80002000000,
>     vaddr=<value optimized out>, fault_type=1 '\001', fault_flags=0)
>     at /usr/src/sys/vm/vm_fault.c:273
> #6  0xffffffff80d5612c in trap_pfault (frame=0xfffffe0c56854320,
> usermode=0)
>     at /usr/src/sys/amd64/amd64/trap.c:757
> #7  0xffffffff80d558fa in trap (frame=0xfffffe0c56854320)
>     at /usr/src/sys/amd64/amd64/trap.c:447
> #8  0xffffffff80d3b8d2 in calltrap ()
>     at /usr/src/sys/amd64/amd64/exception.S:236
> #9  0xffffffff81a494c5 in zap_leaf_array_create (l=0xfffff8008b3a3b00,
>     buf=<value optimized out>, integer_size=1,
>     num_integers=<value optimized out>)
> ---Type <return> to continue, or q <return> to quit---
>     at
> /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zap_leaf.c:198
> #10 0xffffffff81a49a5a in zap_entry_create (l=0xfffff8008b3a3b00,
>     zn=0xfffff8008bf19e00, cd=Cannot access memory at address 0x0
> )
>     at
> /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zap_leaf.c:644
> #11 0xffffffff81a45eee in fzap_add_cd (zn=0xfffff8008bf19e00,
>     integer_size=<value optimized out>, num_integers=1,
>     val=0xfffffe0c568546d0, cd=4294967295, tx=0xfffff80787758c00)
>     at
> /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c:814
> #12 0xffffffff81a4c051 in zap_add (os=<value optimized out>,
>     zapobj=<value optimized out>, key=<value optimized out>,
>     integer_size=<value optimized out>, num_integers=<value optimized out>,
>     val=<value optimized out>, tx=<value optimized out>)
>     at
> /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zap_micro.c:1014
> #13 0xffffffff81a6bfb5 in zfs_link_create (dl=0xfffff800966cce00,
>     zp=<value optimized out>, tx=0xfffff80787758c00,
>     flag=<value optimized out>)
>     at
> /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_dir.c:767
> #14 0xffffffff81a87a34 in zfs_freebsd_rename (ap=<value optimized out>)
> ---Type <return> to continue, or q <return> to quit---
>     at
> /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:4067
> #15 0xffffffff80e81e1b in VOP_RENAME_APV (vop=<value optimized out>,
>     a=<value optimized out>) at vnode_if.c:1546
> #16 0xffffffff80a03476 in kern_renameat (td=<value optimized out>,
>     oldfd=<value optimized out>, old=<value optimized out>,
>     newfd=<value optimized out>, new=<value optimized out>,
>     pathseg=<value optimized out>) at vnode_if.h:636
> #17 0xffffffff80d5694f in amd64_syscall (td=0xfffff8048f7fd000, traced=0)
>     at subr_syscall.c:141
> #18 0xffffffff80d3bbbb in Xfast_syscall ()
>     at /usr/src/sys/amd64/amd64/exception.S:396
> #19 0x000000080381fbaa in ?? ()
> Previous frame inner to this frame (corrupt stack?)
> Current language:  auto; currently minimal
> (kgdb) frame 9
> #9  0xffffffff81a494c5 in zap_leaf_array_create (l=0xfffff8008b3a3b00,
>     buf=<value optimized out>, integer_size=1,
>     num_integers=<value optimized out>)
>     at
> /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zap_leaf.c:198
> 198                 ZAP_LEAF_CHUNK(l, chunk).l_free.lf_next;
> (kgdb) p chunk
> No symbol "chunk" in current context.
>
> ... now I don't debug kernels too often, but I don't know why I can't see
> chunk.  It's declared as an int in this function.
>
> Does this give anyone insight?
>
>