FreeBSD.org IPv6 issue - AAAA records disabled

[Ivan Voras]

On 10 May 2010 17:02, Andrew Gallatin <gallatin at cs.duke.edu> wrote:
> Ivan Voras wrote:

> In the case of 143046, he says that disabling IPv6 support in his
> jails solved the problem.  He also saw the panic:sbdrop, as well
> as some others which makes it seem like something is holding a
> pointer to a free mbuf and modifying it behind the back of the
> new owner.  He saw this most often:
>
> Tracing pid 12 tid 100063 td 0xffffff00092c7390
> mb_free_ext() at mb_free_ext+0x15
> m_freem() at m_freem+0x23
> ether_input() at ether_input+0x56
> mxge_intr() at mxge_intr+0x5b2
> intr_event_execute_handlers() at intr_event_execute_handlers+0x132
> ithread_loop() at ithread_loop+0x7d
> fork_exit() at fork_exit+0x121
> fork_trampoline() at fork_trampoline+0xe
>
> This is coming from the "discard frame w/o packet header"
> codepath in ether_input.  With the way I stock my rings,
> this is impossible unless something else is scribbling on
> my mbufs.

I have a different trace but the dmesg buffer on crashed machines
usually contains the message
"em0: discard frame w/o packet header"
just before the panic so this still looks like it.

Here are my most usual traces:

#1  0xffffffff8058d179 in boot (howto=260)
    at /usr/src/sys/kern/kern_shutdown.c:416
#2  0xffffffff8058d5ac in panic (
    fmt=0xffffffff80971c38 "%s: sockbuf %p and mbuf %p clashing")
    at /usr/src/sys/kern/kern_shutdown.c:579
#3  0xffffffff805ea6d4 in sbsndptr (sb=NA)
    at /usr/src/sys/kern/uipc_sockbuf.c:954
#4  0xffffffff806ff812 in tcp_output (tp=0xffffff0001ae46e0)
    at /usr/src/sys/netinet/tcp_output.c:814
#5  0xffffffff8070c274 in tcp_usr_send (so=0xffffff0001f7ed48, flags=0, m=NA)
    at tcp_offload.h:282
#6  0xffffffff805f05b6 in sosend_generic (so=0xffffff0001f7ed48, addr=0x0,
    uio=0x0, top=0xffffff0001909c00, control=0x0, flags=NA)
    at /usr/src/sys/kern/uipc_socket.c:1256
#7  0xffffffff8077c93f in svc_vc_reply (xprt=0xffffff00018ba000, msg=NA)
    at /usr/src/sys/rpc/svc_vc.c:732
#8  0xffffffff8077851a in svc_sendreply_common (rqstp=0xffffff00425f2800,
    rply=0xffffff8070811990, body=0xffffff006c70f000)
    at /usr/src/sys/rpc/svc.c:538
#9  0xffffffff807793b9 in svc_sendreply_mbuf (NA)
    at /usr/src/sys/rpc/svc.c:594
#10 0xffffffff80760669 in nfssvc_program (rqst=0xffffff00425f2800, xprt=NA)
    at /usr/src/sys/nfsserver/nfs_srvkrpc.c:371
#11 0xffffffff80778f42 in svc_run_internal (pool=0xffffff00016cea00,
    ismaster=0) at /usr/src/sys/rpc/svc.c:893
#12 0xffffffff807792bb in svc_thread_start (arg=NA)
    at /usr/src/sys/rpc/svc.c:1198
#13 0xffffffff80564b78 in fork_exit (
    callout=0xffffffff807792b0 <svc_thread_start>, arg=0xffffff00016cea00,
    frame=0xffffff8070811c80) at /usr/src/sys/kern/kern_fork.c:843
#14 0xffffffff808594ae in fork_trampoline ()
    at /usr/src/sys/amd64/amd64/exception.S:561

(Note: NFS, for what it's worth) And:

#1  0xffffffff805a2a69 in boot (howto=260)
    at /usr/src/sys/kern/kern_shutdown.c:416
#2  0xffffffff805a2e9c in panic (fmt=0xffffffff8098f15f "sbdrop")
    at /usr/src/sys/kern/kern_shutdown.c:579
#3  0xffffffff80603a53 in sbdrop_internal (sb=NA)
    at /usr/src/sys/kern/uipc_sockbuf.c:858
#4  0xffffffff80710ae7 in tcp_do_segment (m=0xffffff0001941500,
    th=0xffffff000194157c, so=0xffffff0032efaaa0, tp=0xffffff00ddc7f000,
    drop_hdrlen=52, tlen=14, iptos=0 '\0', ti_locked=2)
    at /usr/src/sys/netinet/tcp_input.c:2355
#5  0xffffffff807128d2 in tcp_input (m=0xffffff0001941500, off0=NA)
    at /usr/src/sys/netinet/tcp_input.c:1020
#6  0xffffffff806ae23a in ip_input (m=0xffffff0001941500)
    at /usr/src/sys/netinet/ip_input.c:804
#7  0xffffffff8065b30d in swi_net (arg=NA) at /usr/src/sys/net/netisr.c:716
#8  0xffffffff8057bf8d in intr_event_execute_handlers (p=NA)
    at /usr/src/sys/kern/kern_intr.c:1220
#9  0xffffffff8057d63e in ithread_loop (arg=0xffffff00014b66c0)
    at /usr/src/sys/kern/kern_intr.c:1233
#10 0xffffffff80579f58 in fork_exit (
    callout=0xffffffff8057d5b0 <ithread_loop>, arg=0xffffff00014b66c0,
    frame=0xffffff8000044c80) at /usr/src/sys/kern/kern_fork.c:843
#11 0xffffffff8086f3ae in fork_trampoline ()
    at /usr/src/sys/amd64/amd64/exception.S:561
#12 0x0000000000000000 in ?? ()

(Looks like a regular input path)

> I think something may be holding onto an mbuf after free,
> then re-freeing it.  But only after somebody else allocated
> it.   I was hoping that the mbuf double free referenced
> above was the smoking gun, but it turns out that there isn't
> even a bge interface in my pr (just bce and mxge).

Since I have an em it looks interface-agnostic.

Also, moving this to freebsd-net at .