Panic on -stable with 10 Gigabit driver

[Prafulla Deuskar]

All,

We are having a reproducible panic with our 10 Gigabit driver on -stable under
stress. (4.8-RELEASE, 4.9-RELEASE)
We have not been able to reproduce the panic on 5.1-RELEASE with same load.

The test is nttcp with 1500 byte MTU.

If anyone on the list has any ideas please let me know.

Thanks,
Prafulla

Here is the crash dump output:
------------------------------

#0  dumpsys () at /usr/src/sys/kern/kern_shutdown.c:487
487             if (dumping++) {
(kgdb) where
#0  dumpsys () at /usr/src/sys/kern/kern_shutdown.c:487
#1  0xc021a823 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:316
#2  0xc021ac61 in panic (fmt=0xc03b1944 "from debugger")
    at /usr/src/sys/kern/kern_shutdown.c:595
#3  0xc014be59 in db_panic (addr=-1071421816, have_addr=0, count=-1,
    modif=0xc0417874 "") at /usr/src/sys/ddb/db_command.c:435
#4  0xc014bdf9 in db_command (last_cmdp=0xc041daa4, cmd_table=0xc041d8e4,
    aux_cmd_tablep=0xc0460d98) at /usr/src/sys/ddb/db_command.c:333
#5  0xc014bebe in db_command_loop () at /usr/src/sys/ddb/db_command.c:457
#6  0xc014dffb in db_trap (type=12, code=0) at /usr/src/sys/ddb/db_trap.c:71
#7  0xc0379c8e in kdb_trap (type=12, code=0, regs=0xc04179c8)
    at /usr/src/sys/i386/i386/db_interface.c:158
#8  0xc038a4a4 in trap_fatal (frame=0xc04179c8, eva=117471232)
    at /usr/src/sys/i386/i386/trap.c:969
#9  0xc038a17d in trap_pfault (frame=0xc04179c8, usermode=0, eva=117471232)
    at /usr/src/sys/i386/i386/trap.c:867
#10 0xc0389cf7 in trap (frame={tf_fs = -1069481968, tf_es = -1069481968, tf_ds = 16,
      tf_edi = 6756506, tf_esi = 1, tf_ebp = -1069450724, tf_isp = -1069450764,
      tf_ebx = -1039395840, tf_edx = -1039732736, tf_ecx = 117471232, tf_eax = 565021,
      tf_trapno = 12, tf_err = 0, tf_eip = -1071421816, tf_cs = 8, tf_eflags = 66050,
      tf_esp = 0, tf_ss = -955602944}) at /usr/src/sys/i386/i386/trap.c:466
#11 0xc0236688 in m_getcl (how=1, type=1, flags=2) at /usr/src/sys/kern/uipc_mbuf.c:589
#12 0xc03a4282 in ixgb_get_buf (i=254, adapter=0xc70aa800, nmp=0x0)
    at /usr/src/sys/dev/ixgb/if_ixgb.c:1768
#13 0xc03a495d in ixgb_process_receive_interrupts (adapter=0xc70aa800, count=-35)
    at /usr/src/sys/dev/ixgb/if_ixgb.c:2095
#14 0xc03a310e in ixgb_intr (arg=0xc70aa800) at /usr/src/sys/dev/ixgb/if_ixgb.c:798
#15 0xc0394cf9 in intr_mux (arg=0xc70acfe0) at /usr/src/sys/i386/isa/intr_machdep.c:582
#16 0xc0380e02 in cpu_idle () at /usr/src/sys/i386/i386/machdep.c:1022
(kgdb) up 10
#10 0xc0389cf7 in trap (frame={tf_fs = -1069481968, tf_es = -1069481968, tf_ds = 16,
      tf_edi = 6756506, tf_esi = 1, tf_ebp = -1069450724, tf_isp = -1069450764,
      tf_ebx = -1039395840, tf_edx = -1039732736, tf_ecx = 117471232, tf_eax = 565021,
      tf_trapno = 12, tf_err = 0, tf_eip = -1071421816, tf_cs = 8, tf_eflags = 66050,
      tf_esp = 0, tf_ss = -955602944}) at /usr/src/sys/i386/i386/trap.c:466
466                             (void) trap_pfault(&frame, FALSE, eva);
(kgdb) frame frame->tf_ebp frame->tf_eip
#0  0xc0236688 in m_getcl (how=1, type=1, flags=2) at /usr/src/sys/kern/uipc_mbuf.c:589
589                     MCLGET(mp, how);
(kgdb) list
584                     } else
585                             MGETHDR(mp, how, type);
586             } else
587                     MGET(mp, how, type);
588             if (mp) {
589                     MCLGET(mp, how);
590                     if ( (mp->m_flags & M_EXT) == 0) {
591                             m_free(mp);
592                             mp = NULL;
593                     }
(kgdb) print mp
$1 = (struct mbuf *) 0xc20c1400
(kgdb) print how
$2 = 1
(kgdb) up
#1  0xc03a4282 in ixgb_get_buf (i=254, adapter=0xc70aa800, nmp=0x0)
    at /usr/src/sys/dev/ixgb/if_ixgb.c:1768
1768                    mp->m_next = NULL;
(kgdb) list
1763                    }
1764                    mp->m_len = mp->m_pkthdr.len = MCLBYTES;
1765            } else {
1766                    mp->m_len = mp->m_pkthdr.len = MCLBYTES;
1767                    mp->m_data = mp->m_ext.ext_buf;
1768                    mp->m_next = NULL;
1769            }
1770
1771            if (ifp->if_mtu <= ETHERMTU) {
1772                    m_adj(mp, ETHER_ALIGN);