9/STABLE Panic at netisr_dispatch_src w/ em(4) + PF

Nick Rogers ncrogers at gmail.com
Fri May 16 14:54:40 UTC 2014


On Fri, May 2, 2014 at 1:49 PM, Nick Rogers <ncrogers at gmail.com> wrote:
> Hello,
>
> I am hoping someone can help me debug a kernel panic I have been experiencing
> on one of my production systems. The system is a PF+ALTQ firewall/gateway with
> about 1k simultaneous devices behind it at any given time, pushing no more
> than 100Mb/s of traffic. I have obtained a crash dump and tried to debug it
> with kgdb, but am still at a loss.
>
> I have completely replaced the hardware to rule out issues with
> disk/memory/etc, and it appears to be a kernel issue, likely with e1000 driver
> and/or PF.
>
> The panic seems to happen during times of heavier use, but the frequency is
> not very predictable (anywhere from a few times a day to a once a week), so I
> feel like its some kind of strange traffic pattern that I am unable to
> pinpoint.
>
> I am using a slightly custom kernel based on GENERIC, mainly to bring in ALTQ
> and some other options.
>
> It may be worth noting that I also have IGB_LEGACY_TX set for the e1000
> driver, although I do not believe that should affect em(4).
>
> Hoping someone can be of assistance in debugging this problem. I am willing to
> test patches and pull in the e1000 driver from HEAD or newer 9/STABLE if that
> is advisable. Unfortunately I cannot try 10-STABLE.

There is now a PR for this. It appears the issue may be a bug in 9.x
PF based on the responses to the PR thus far.

http://www.freebsd.org/cgi/query-pr.cgi?pr=189741

>
> Thanks,
>
> -Nick Rogers
>
>
> uname -v
> FreeBSD 9.2-STABLE #16 r264331M: Thu Apr 10 21:23:18 EDT 2014
> root at fbsd_91_amd64_builder.rgnets.com:/usr/obj/usr/src/sys/RGNETS
>
>
>
> Here is the kernel conf...
> ..............................................................................
> include GENERIC
>
> ident RGNETS
>
> makeoptions MODULES_OVERRIDE=""
>
> options DEVICE_POLLING
>
> device crypto
> device cryptodev
>
> options VLAN_ARRAY
>
> device amdtemp
>
> # PF
> device pf
> device pflog
> options ALTQ
> options ALTQ_CBQ        # Class Bases Queuing (CBQ)
> options ALTQ_RED        # Random Early Detection (RED)
> options ALTQ_RIO        # RED In/Out
> options ALTQ_HFSC       # Hierarchical Packet Scheduler (HFSC)
> options ALTQ_PRIQ       # Priority Queuing (PRIQ)
> options ALTQ_NOPCC      # Required for SMP build
>
> # PPPoE
> options NETGRAPH
> options NETGRAPH_ETHER
> options NETGRAPH_PPPOE
> options NETGRAPH_SOCKET
>
> # IPsec
> device enc
> options IPSEC
> options IPSEC_FILTERTUNNEL
> options IPSEC_NAT_T
> ..............................................................................
>
>
> The crash dump....
> ..............................................................................
>
> GNU gdb 6.1.1 [FreeBSD]
> Copyright 2004 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and you are
> welcome to change it and/or distribute copies of it under certain conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB.  Type "show warranty" for details.
> This GDB was configured as "amd64-marcel-freebsd"...
>
> Unread portion of the kernel message buffer:
>
>
> Fatal trap 12: page fault while in kernel mode
> cpuid = 5; apic id = 05
> fault virtual address = 0x10
> fault code = supervisor read data, page not present
> instruction pointer = 0x20:0xffffffff8033d350
> stack pointer        = 0x28:0xffffff83545384b0
> frame pointer        = 0x28:0xffffff83545384c0
> code segment = base 0x0, limit 0xfffff, type 0x1b
> = DPL 0, pres 1, long 1, def32 0, gran 1
> processor eflags = interrupt enabled, resume, IOPL = 0
> current process = 12 (irq262: em2:rx 0)
> trap number = 12
> panic: page fault
> cpuid = 5
> KDB: stack backtrace:
> #0 0xffffffff80956836 at kdb_backtrace+0x66
> #1 0xffffffff8091c40e at panic+0x1ce
> #2 0xffffffff80d31e70 at trap_fatal+0x290
> #3 0xffffffff80d321d1 at trap_pfault+0x211
> #4 0xffffffff80d327d3 at trap+0x363
> #5 0xffffffff80d1b9d3 at calltrap+0x8
> #6 0xffffffff8034872d at pf_test_rule+0x17ed
> #7 0xffffffff8034ba12 at pf_test+0x1032
> #8 0xffffffff8035112b at pf_check_in+0x2b
> #9 0xffffffff809e952e at pfil_run_hooks+0x9e
> #10 0xffffffff80a5286a at ip_input+0x2ea
> #11 0xffffffff809e8858 at netisr_dispatch_src+0x218
> #12 0xffffffff809df93d at ether_demux+0x14d
> #13 0xffffffff809dfc1e at ether_nh_input+0x1fe
> #14 0xffffffff809e8858 at netisr_dispatch_src+0x218
> #15 0xffffffff809df85f at ether_demux+0x6f
> #16 0xffffffff809dfc1e at ether_nh_input+0x1fe
> #17 0xffffffff809e8858 at netisr_dispatch_src+0x218
> Uptime: 17d7h20m59s
> Dumping 2932 out of 12256 MB: (CTRL-C to abort)
> ..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%
>
> Reading symbols from /boot/kernel/aio.ko...Reading symbols from
> /boot/kernel/aio.ko.symbols...done.
> done.
> Loaded symbols for /boot/kernel/aio.ko
> Reading symbols from /boot/kernel/coretemp.ko...Reading symbols from
> /boot/kernel/coretemp.ko.symbols...done.
> done.
> Loaded symbols for /boot/kernel/coretemp.ko
> Reading symbols from /boot/kernel/cc_htcp.ko...Reading symbols from
> /boot/kernel/cc_htcp.ko.symbols...done.
> done.
> Loaded symbols for /boot/kernel/cc_htcp.ko
> #0  doadump (textdump=Variable "textdump" is not available.
> ) at pcpu.h:234
> 234 pcpu.h: No such file or directory.
> in pcpu.h
> (kgdb) list *0xffffffff8033d350
> 0xffffffff8033d350 is in pf_addrcpy (/usr/src/sys/contrib/pf/net/pf.c:512).
> 507 pf_addrcpy(struct pf_addr *dst, struct pf_addr *src, sa_family_t af)
> 508 {
> 509 switch (af) {
> 510 #ifdef INET
> 511 case AF_INET:
> 512 dst->addr32[0] = src->addr32[0];
> 513 break;
> 514 #endif /* INET */
> 515 case AF_INET6:
> 516 dst->addr32[0] = src->addr32[0];
> (kgdb) backtrace
> #0  doadump (textdump=Variable "textdump" is not available.
> ) at pcpu.h:234
> #1  0xffffffff8091bee6 in kern_reboot (howto=260) at
> /usr/src/sys/kern/kern_shutdown.c:454
> #2  0xffffffff8091c3e7 in panic (fmt=0x1 <Address 0x1 out of bounds>)
> at /usr/src/sys/kern/kern_shutdown.c:642
> #3  0xffffffff80d31e70 in trap_fatal (frame=0xc, eva=Variable "eva" is
> not available.
> ) at /usr/src/sys/amd64/amd64/trap.c:878
> #4  0xffffffff80d321d1 in trap_pfault (frame=0xffffff8354538400,
> usermode=0) at /usr/src/sys/amd64/amd64/trap.c:794
> #5  0xffffffff80d327d3 in trap (frame=0xffffff8354538400) at
> /usr/src/sys/amd64/amd64/trap.c:456
> #6  0xffffffff80d1b9d3 in calltrap () at
> /usr/src/sys/amd64/amd64/exception.S:232
> #7  0xffffffff8033d350 in pf_addrcpy (dst=0xfffffe010c6416b8,
> src=0x10, af=2 '\002') at /usr/src/sys/contrib/pf/net/pf.c:522
> #8  0xffffffff8034872d in pf_test_rule (rm=0xffffff8354538788,
> sm=0xffffff8354538780, direction=1, kif=0xfffffe0007d08100,
> m=0xfffffe0030555d00, off=20, h=0xfffffe0030bad00e,
>     pd=0xffffff83545386c0, am=0xffffff8354538790,
> rsm=0xffffff8354538778, ifq=0x0, inp=0x0) at
> /usr/src/sys/contrib/pf/net/pf.c:3900
> #9  0xffffffff8034ba12 in pf_test (dir=1, ifp=Variable "ifp" is not available.
> ) at /usr/src/sys/contrib/pf/net/pf.c:6776
> #10 0xffffffff8035112b in pf_check_in (arg=Variable "arg" is not available.
> ) at /usr/src/sys/contrib/pf/net/pf_ioctl.c:4131
> #11 0xffffffff809e952e in pfil_run_hooks (ph=Variable "ph" is not available.
> ) at /usr/src/sys/net/pfil.c:82
> #12 0xffffffff80a5286a in ip_input (m=0xfffffe0030555d00) at
> /usr/src/sys/netinet/ip_input.c:510
> #13 0xffffffff809e8858 in netisr_dispatch_src (proto=1,
> source=Variable "source" is not available.
> ) at /usr/src/sys/net/netisr.c:1013
> #14 0xffffffff809df93d in ether_demux (ifp=0xfffffe0030239000,
> m=0xfffffe0030555d00) at /usr/src/sys/net/if_ethersubr.c:943
> #15 0xffffffff809dfc1e in ether_nh_input (m=Variable "m" is not available.
> ) at /usr/src/sys/net/if_ethersubr.c:762
> #16 0xffffffff809e8858 in netisr_dispatch_src (proto=9,
> source=Variable "source" is not available.
> ) at /usr/src/sys/net/netisr.c:1013
> #17 0xffffffff809df85f in ether_demux (ifp=0xfffffe0003f0c800,
> m=0xfffffe0030555d00) at /usr/src/sys/net/if_ethersubr.c:852
> #18 0xffffffff809dfc1e in ether_nh_input (m=Variable "m" is not available.
> ) at /usr/src/sys/net/if_ethersubr.c:762
> #19 0xffffffff809e8858 in netisr_dispatch_src (proto=9,
> source=Variable "source" is not available.
> ) at /usr/src/sys/net/netisr.c:1013
> #20 0xffffffff804ccb58 in em_rxeof (rxr=0xfffffe0007308200, count=-2,
> done=0x0) at /usr/src/sys/dev/e1000/if_em.c:4525
> #21 0xffffffff804cceb6 in em_msix_rx (arg=Variable "arg" is not available.
> ) at /usr/src/sys/dev/e1000/if_em.c:1593
> #22 0xffffffff808ecb1d in intr_event_execute_handlers (p=Variable "p"
> is not available.
> ) at /usr/src/sys/kern/kern_intr.c:1272
> #23 0xffffffff808ee30d in ithread_loop (arg=0xfffffe000730c300) at
> /usr/src/sys/kern/kern_intr.c:1285
> #24 0xffffffff808e951f in fork_exit (callout=0xffffffff808ee270
> <ithread_loop>, arg=0xfffffe000730c300, frame=0xffffff8354538c40) at
> /usr/src/sys/kern/kern_fork.c:996
> #25 0xffffffff80d1befe in fork_trampoline () at
> /usr/src/sys/amd64/amd64/exception.S:606
> #26 0x0000000000000000 in ?? ()
> #27 0x0000000000000000 in ?? ()
> #28 0x0000000000000001 in ?? ()
> #29 0x0000000000000000 in ?? ()
> #30 0x0000000000000000 in ?? ()
> #31 0x0000000000000000 in ?? ()
> #32 0x0000000000000000 in ?? ()
> #33 0x0000000000000000 in ?? ()
> #34 0x0000000000000000 in ?? ()
> #35 0x0000000000000000 in ?? ()
> #36 0x0000000000000000 in ?? ()
> #37 0x0000000000000000 in ?? ()
> #38 0x0000000000000000 in ?? ()
> #39 0x0000000000000000 in ?? ()
> #40 0x0000000000000000 in ?? ()
> #41 0x0000000000000000 in ?? ()
> #42 0x0000000000000000 in ?? ()
> #43 0x0000000000000000 in ?? ()
> #44 0x0000000000000000 in ?? ()
> #45 0x0000000000000000 in ?? ()
> #46 0x0000000000000000 in ?? ()
> #47 0x0000000000000000 in ?? ()
> #48 0x0000000000000000 in ?? ()
> ---Type <return> to continue, or q <return> to quit---
> #49 0x0000000000000000 in ?? ()
> #50 0x0000000000000000 in ?? ()
> #51 0xfffffe0007304920 in ?? ()
> #52 0xfffffe0003afd000 in ?? ()
> #53 0xfffffe0007304920 in ?? ()
> #54 0xffffff8354538b40 in ?? ()
> #55 0xffffff8354538ae8 in ?? ()
> #56 0xfffffe0003b00920 in ?? ()
> #57 0xffffffff80948646 in sched_switch (td=0xffffffff808ee270,
> newtd=0xfffffe000730c300, flags=Variable "flags" is not available.
> ) at /usr/src/sys/kern/sched_ule.c:1898
> Previous frame inner to this frame (corrupt stack?)
>
> ..............................................................................
>
>
>
> Relevant pciconf output...
> ..............................................................................
>
> em0 at pci0:1:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086 rev=0x00 hdr=0x00
>     vendor     = 'Intel Corporation'
>     device     = '82574L Gigabit Network Connection'
>     class      = network
>     subclass   = ethernet
>     cap 01[c8] = powerspec 2  supports D0 D3  current D0
>     cap 05[d0] = MSI supports 1 message, 64 bit
>     cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1)
>     cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled
> ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected
> em1 at pci0:2:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086 rev=0x00 hdr=0x00
>     vendor     = 'Intel Corporation'
>     device     = '82574L Gigabit Network Connection'
>     class      = network
>     subclass   = ethernet
>     cap 01[c8] = powerspec 2  supports D0 D3  current D0
>     cap 05[d0] = MSI supports 1 message, 64 bit
>     cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1)
>     cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled
> ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected
> em2 at pci0:7:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086 rev=0x00 hdr=0x00
>     vendor     = 'Intel Corporation'
>     device     = '82574L Gigabit Network Connection'
>     class      = network
>     subclass   = ethernet
>     cap 01[c8] = powerspec 2  supports D0 D3  current D0
>     cap 05[d0] = MSI supports 1 message, 64 bit
>     cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1)
>     cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled
> ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected
> em3 at pci0:8:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086 rev=0x00 hdr=0x00
>     vendor     = 'Intel Corporation'
>     device     = '82574L Gigabit Network Connection'
>     class      = network
>     subclass   = ethernet
>     cap 01[c8] = powerspec 2  supports D0 D3  current D0
>     cap 05[d0] = MSI supports 1 message, 64 bit
>     cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1)
>     cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled
>
> ..............................................................................
>
>
> dev.em sysctl....
> ..............................................................................
>
> dev.em.0.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
> dev.em.0.%driver: em
> dev.em.0.%location: slot=0 function=0
> dev.em.0.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9
> subdevice=0x10d3 class=0x020000
> dev.em.0.%parent: pci1
> dev.em.0.nvm: -1
> dev.em.0.debug: -1
> dev.em.0.fc: 3
> dev.em.0.rx_int_delay: 0
> dev.em.0.tx_int_delay: 66
> dev.em.0.rx_abs_int_delay: 66
> dev.em.0.tx_abs_int_delay: 66
> dev.em.0.itr: 488
> dev.em.0.rx_processing_limit: -1
> dev.em.0.eee_control: 1
> dev.em.0.link_irq: 2
> dev.em.0.mbuf_alloc_fail: 0
> dev.em.0.cluster_alloc_fail: 0
> dev.em.0.dropped: 0
> dev.em.0.tx_dma_fail: 0
> dev.em.0.rx_overruns: 0
> dev.em.0.watchdog_timeouts: 0
> dev.em.0.device_control: 1477444168
> dev.em.0.rx_control: 67141634
> dev.em.0.fc_high_water: 18432
> dev.em.0.fc_low_water: 16932
> dev.em.0.queue0.txd_head: 3265
> dev.em.0.queue0.txd_tail: 3265
> dev.em.0.queue0.tx_irq: 81153071
> dev.em.0.queue0.no_desc_avail: 0
> dev.em.0.queue0.rxd_head: 388
> dev.em.0.queue0.rxd_tail: 387
> dev.em.0.queue0.rx_irq: 79015024
> dev.em.0.mac_stats.excess_coll: 0
> dev.em.0.mac_stats.single_coll: 0
> dev.em.0.mac_stats.multiple_coll: 0
> dev.em.0.mac_stats.late_coll: 0
> dev.em.0.mac_stats.collision_count: 0
> dev.em.0.mac_stats.symbol_errors: 0
> dev.em.0.mac_stats.sequence_errors: 0
> dev.em.0.mac_stats.defer_count: 0
> dev.em.0.mac_stats.missed_packets: 0
> dev.em.0.mac_stats.recv_no_buff: 0
> dev.em.0.mac_stats.recv_undersize: 0
> dev.em.0.mac_stats.recv_fragmented: 0
> dev.em.0.mac_stats.recv_oversize: 0
> dev.em.0.mac_stats.recv_jabber: 0
> dev.em.0.mac_stats.recv_errs: 0
> dev.em.0.mac_stats.crc_errs: 0
> dev.em.0.mac_stats.alignment_errs: 0
> dev.em.0.mac_stats.coll_ext_errs: 0
> dev.em.0.mac_stats.xon_recvd: 6
> dev.em.0.mac_stats.xon_txd: 0
> dev.em.0.mac_stats.xoff_recvd: 6
> dev.em.0.mac_stats.xoff_txd: 0
> dev.em.0.mac_stats.total_pkts_recvd: 122072630
> dev.em.0.mac_stats.good_pkts_recvd: 122072618
> dev.em.0.mac_stats.bcast_pkts_recvd: 257
> dev.em.0.mac_stats.mcast_pkts_recvd: 0
> dev.em.0.mac_stats.rx_frames_64: 8634
> dev.em.0.mac_stats.rx_frames_65_127: 67656673
> dev.em.0.mac_stats.rx_frames_128_255: 714152
> dev.em.0.mac_stats.rx_frames_256_511: 609615
> dev.em.0.mac_stats.rx_frames_512_1023: 8646536
> dev.em.0.mac_stats.rx_frames_1024_1522: 44437008
> dev.em.0.mac_stats.good_octets_recvd: 82940411216
> dev.em.0.mac_stats.good_octets_txd: 25718335997
> dev.em.0.mac_stats.total_pkts_txd: 99833592
> dev.em.0.mac_stats.good_pkts_txd: 99833592
> dev.em.0.mac_stats.bcast_pkts_txd: 13
> dev.em.0.mac_stats.mcast_pkts_txd: 0
> dev.em.0.mac_stats.tx_frames_64: 2193
> dev.em.0.mac_stats.tx_frames_65_127: 29089783
> dev.em.0.mac_stats.tx_frames_128_255: 54412030
> dev.em.0.mac_stats.tx_frames_256_511: 9565246
> dev.em.0.mac_stats.tx_frames_512_1023: 1080398
> dev.em.0.mac_stats.tx_frames_1024_1522: 5683942
> dev.em.0.mac_stats.tso_txd: 1468623
> dev.em.0.mac_stats.tso_ctx_fail: 0
> dev.em.0.interrupts.asserts: 2
> dev.em.0.interrupts.rx_pkt_timer: 0
> dev.em.0.interrupts.rx_abs_timer: 0
> dev.em.0.interrupts.tx_pkt_timer: 0
> dev.em.0.interrupts.tx_abs_timer: 0
> dev.em.0.interrupts.tx_queue_empty: 0
> dev.em.0.interrupts.tx_queue_min_thresh: 0
> dev.em.0.interrupts.rx_desc_min_thresh: 0
> dev.em.0.interrupts.rx_overrun: 0
> dev.em.1.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
> dev.em.1.%driver: em
> dev.em.1.%location: slot=0 function=0
> dev.em.1.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9
> subdevice=0x10d3 class=0x020000
> dev.em.1.%parent: pci2
> dev.em.1.nvm: -1
> dev.em.1.debug: -1
> dev.em.1.fc: 3
> dev.em.1.rx_int_delay: 0
> dev.em.1.tx_int_delay: 66
> dev.em.1.rx_abs_int_delay: 66
> dev.em.1.tx_abs_int_delay: 66
> dev.em.1.itr: 488
> dev.em.1.rx_processing_limit: -1
> dev.em.1.eee_control: 1
> dev.em.1.link_irq: 2
> dev.em.1.mbuf_alloc_fail: 0
> dev.em.1.cluster_alloc_fail: 0
> dev.em.1.dropped: 0
> dev.em.1.tx_dma_fail: 1
> dev.em.1.rx_overruns: 0
> dev.em.1.watchdog_timeouts: 0
> dev.em.1.device_control: 1477444168
> dev.em.1.rx_control: 67141634
> dev.em.1.fc_high_water: 18432
> dev.em.1.fc_low_water: 16932
> dev.em.1.queue0.txd_head: 2451
> dev.em.1.queue0.txd_tail: 2453
> dev.em.1.queue0.tx_irq: 143904807
> dev.em.1.queue0.no_desc_avail: 0
> dev.em.1.queue0.rxd_head: 342
> dev.em.1.queue0.rxd_tail: 341
> dev.em.1.queue0.rx_irq: 159303310
> dev.em.1.mac_stats.excess_coll: 0
> dev.em.1.mac_stats.single_coll: 0
> dev.em.1.mac_stats.multiple_coll: 0
> dev.em.1.mac_stats.late_coll: 0
> dev.em.1.mac_stats.collision_count: 0
> dev.em.1.mac_stats.symbol_errors: 0
> dev.em.1.mac_stats.sequence_errors: 0
> dev.em.1.mac_stats.defer_count: 0
> dev.em.1.mac_stats.missed_packets: 0
> dev.em.1.mac_stats.recv_no_buff: 0
> dev.em.1.mac_stats.recv_undersize: 0
> dev.em.1.mac_stats.recv_fragmented: 0
> dev.em.1.mac_stats.recv_oversize: 0
> dev.em.1.mac_stats.recv_jabber: 0
> dev.em.1.mac_stats.recv_errs: 0
> dev.em.1.mac_stats.crc_errs: 0
> dev.em.1.mac_stats.alignment_errs: 0
> dev.em.1.mac_stats.coll_ext_errs: 0
> dev.em.1.mac_stats.xon_recvd: 1
> dev.em.1.mac_stats.xon_txd: 0
> dev.em.1.mac_stats.xoff_recvd: 1
> dev.em.1.mac_stats.xoff_txd: 0
> dev.em.1.mac_stats.total_pkts_recvd: 331901758
> dev.em.1.mac_stats.good_pkts_recvd: 331901756
> dev.em.1.mac_stats.bcast_pkts_recvd: 13467
> dev.em.1.mac_stats.mcast_pkts_recvd: 0
> dev.em.1.mac_stats.rx_frames_64: 13905035
> dev.em.1.mac_stats.rx_frames_65_127: 22315178
> dev.em.1.mac_stats.rx_frames_128_255: 8343368
> dev.em.1.mac_stats.rx_frames_256_511: 8602323
> dev.em.1.mac_stats.rx_frames_512_1023: 8170288
> dev.em.1.mac_stats.rx_frames_1024_1522: 270565564
> dev.em.1.mac_stats.good_octets_recvd: 420794715670
> dev.em.1.mac_stats.good_octets_txd: 45361473880
> dev.em.1.mac_stats.total_pkts_txd: 217852588
> dev.em.1.mac_stats.good_pkts_txd: 217852588
> dev.em.1.mac_stats.bcast_pkts_txd: 6
> dev.em.1.mac_stats.mcast_pkts_txd: 0
> dev.em.1.mac_stats.tx_frames_64: 64102191
> dev.em.1.mac_stats.tx_frames_65_127: 120705475
> dev.em.1.mac_stats.tx_frames_128_255: 6009336
> dev.em.1.mac_stats.tx_frames_256_511: 4593595
> dev.em.1.mac_stats.tx_frames_512_1023: 4295623
> dev.em.1.mac_stats.tx_frames_1024_1522: 18146368
> dev.em.1.mac_stats.tso_txd: 291134
> dev.em.1.mac_stats.tso_ctx_fail: 0
> dev.em.1.interrupts.asserts: 2
> dev.em.1.interrupts.rx_pkt_timer: 0
> dev.em.1.interrupts.rx_abs_timer: 0
> dev.em.1.interrupts.tx_pkt_timer: 0
> dev.em.1.interrupts.tx_abs_timer: 0
> dev.em.1.interrupts.tx_queue_empty: 0
> dev.em.1.interrupts.tx_queue_min_thresh: 0
> dev.em.1.interrupts.rx_desc_min_thresh: 0
> dev.em.1.interrupts.rx_overrun: 0
> dev.em.2.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
> dev.em.2.%driver: em
> dev.em.2.%location: slot=0 function=0
> dev.em.2.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9
> subdevice=0x10d3 class=0x020000
> dev.em.2.%parent: pci7
> dev.em.2.nvm: -1
> dev.em.2.debug: -1
> dev.em.2.fc: 3
> dev.em.2.rx_int_delay: 0
> dev.em.2.tx_int_delay: 66
> dev.em.2.rx_abs_int_delay: 66
> dev.em.2.tx_abs_int_delay: 66
> dev.em.2.itr: 488
> dev.em.2.rx_processing_limit: -1
> dev.em.2.eee_control: 1
> dev.em.2.link_irq: 1
> dev.em.2.mbuf_alloc_fail: 0
> dev.em.2.cluster_alloc_fail: 0
> dev.em.2.dropped: 0
> dev.em.2.tx_dma_fail: 6823
> dev.em.2.rx_overruns: 0
> dev.em.2.watchdog_timeouts: 0
> dev.em.2.device_control: 1477444168
> dev.em.2.rx_control: 67141634
> dev.em.2.fc_high_water: 18432
> dev.em.2.fc_low_water: 16932
> dev.em.2.queue0.txd_head: 3977
> dev.em.2.queue0.txd_tail: 3977
> dev.em.2.queue0.tx_irq: 220950699
> dev.em.2.queue0.no_desc_avail: 0
> dev.em.2.queue0.rxd_head: 83
> dev.em.2.queue0.rxd_tail: 82
> dev.em.2.queue0.rx_irq: 125920607
> dev.em.2.mac_stats.excess_coll: 0
> dev.em.2.mac_stats.single_coll: 0
> dev.em.2.mac_stats.multiple_coll: 0
> dev.em.2.mac_stats.late_coll: 0
> dev.em.2.mac_stats.collision_count: 0
> dev.em.2.mac_stats.symbol_errors: 0
> dev.em.2.mac_stats.sequence_errors: 0
> dev.em.2.mac_stats.defer_count: 0
> dev.em.2.mac_stats.missed_packets: 0
> dev.em.2.mac_stats.recv_no_buff: 0
> dev.em.2.mac_stats.recv_undersize: 0
> dev.em.2.mac_stats.recv_fragmented: 0
> dev.em.2.mac_stats.recv_oversize: 0
> dev.em.2.mac_stats.recv_jabber: 0
> dev.em.2.mac_stats.recv_errs: 0
> dev.em.2.mac_stats.crc_errs: 0
> dev.em.2.mac_stats.alignment_errs: 0
> dev.em.2.mac_stats.coll_ext_errs: 0
> dev.em.2.mac_stats.xon_recvd: 14123
> dev.em.2.mac_stats.xon_txd: 1
> dev.em.2.mac_stats.xoff_recvd: 14127
> dev.em.2.mac_stats.xoff_txd: 1
> dev.em.2.mac_stats.total_pkts_recvd: 229919303
> dev.em.2.mac_stats.good_pkts_recvd: 229891053
> dev.em.2.mac_stats.bcast_pkts_recvd: 909450
> dev.em.2.mac_stats.mcast_pkts_recvd: 19452
> dev.em.2.mac_stats.rx_frames_64: 1477808
> dev.em.2.mac_stats.rx_frames_65_127: 195114744
> dev.em.2.mac_stats.rx_frames_128_255: 6579690
> dev.em.2.mac_stats.rx_frames_256_511: 5137387
> dev.em.2.mac_stats.rx_frames_512_1023: 4223090
> dev.em.2.mac_stats.rx_frames_1024_1522: 17358334
> dev.em.2.mac_stats.good_octets_recvd: 46129102134
> dev.em.2.mac_stats.good_octets_txd: 419293159496
> dev.em.2.mac_stats.total_pkts_txd: 332661584
> dev.em.2.mac_stats.good_pkts_txd: 332661582
> dev.em.2.mac_stats.bcast_pkts_txd: 48506
> dev.em.2.mac_stats.mcast_pkts_txd: 78
> dev.em.2.mac_stats.tx_frames_64: 14598198
> dev.em.2.mac_stats.tx_frames_65_127: 22287108
> dev.em.2.mac_stats.tx_frames_128_255: 8897511
> dev.em.2.mac_stats.tx_frames_256_511: 9623000
> dev.em.2.mac_stats.tx_frames_512_1023: 8325033
> dev.em.2.mac_stats.tx_frames_1024_1522: 268930732
> dev.em.2.mac_stats.tso_txd: 24357891
> dev.em.2.mac_stats.tso_ctx_fail: 0
> dev.em.2.interrupts.asserts: 2
> dev.em.2.interrupts.rx_pkt_timer: 0
> dev.em.2.interrupts.rx_abs_timer: 0
> dev.em.2.interrupts.tx_pkt_timer: 0
> dev.em.2.interrupts.tx_abs_timer: 0
> dev.em.2.interrupts.tx_queue_empty: 0
> dev.em.2.interrupts.tx_queue_min_thresh: 0
> dev.em.2.interrupts.rx_desc_min_thresh: 0
> dev.em.2.interrupts.rx_overrun: 0
> dev.em.3.%desc: Intel(R) PRO/1000 Network Connection 7.3.8
> dev.em.3.%driver: em
> dev.em.3.%location: slot=0 function=0
> dev.em.3.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9
> subdevice=0x10d3 class=0x020000
> dev.em.3.%parent: pci8
> dev.em.3.nvm: -1
> dev.em.3.debug: -1
> dev.em.3.fc: 3
> dev.em.3.rx_int_delay: 0
> dev.em.3.tx_int_delay: 66
> dev.em.3.rx_abs_int_delay: 66
> dev.em.3.tx_abs_int_delay: 66
> dev.em.3.itr: 488
> dev.em.3.rx_processing_limit: -1
> dev.em.3.eee_control: 1
> dev.em.3.link_irq: 0
> dev.em.3.mbuf_alloc_fail: 0
> dev.em.3.cluster_alloc_fail: 0
> dev.em.3.dropped: 0
> dev.em.3.tx_dma_fail: 0
> dev.em.3.rx_overruns: 0
> dev.em.3.watchdog_timeouts: 0
> dev.em.3.device_control: 1074790984
> dev.em.3.rx_control: 67141634
> dev.em.3.fc_high_water: 18432
> dev.em.3.fc_low_water: 16932
> dev.em.3.queue0.txd_head: 0
> dev.em.3.queue0.txd_tail: 0
> dev.em.3.queue0.tx_irq: 0
> dev.em.3.queue0.no_desc_avail: 0
> dev.em.3.queue0.rxd_head: 0
> dev.em.3.queue0.rxd_tail: 4095
> dev.em.3.queue0.rx_irq: 0
> dev.em.3.mac_stats.excess_coll: 0
> dev.em.3.mac_stats.single_coll: 0
> dev.em.3.mac_stats.multiple_coll: 0
> dev.em.3.mac_stats.late_coll: 0
> dev.em.3.mac_stats.collision_count: 0
> dev.em.3.mac_stats.symbol_errors: 0
> dev.em.3.mac_stats.sequence_errors: 0
> dev.em.3.mac_stats.defer_count: 0
> dev.em.3.mac_stats.missed_packets: 0
> dev.em.3.mac_stats.recv_no_buff: 0
> dev.em.3.mac_stats.recv_undersize: 0
> dev.em.3.mac_stats.recv_fragmented: 0
> dev.em.3.mac_stats.recv_oversize: 0
> dev.em.3.mac_stats.recv_jabber: 0
> dev.em.3.mac_stats.recv_errs: 0
> dev.em.3.mac_stats.crc_errs: 0
> dev.em.3.mac_stats.alignment_errs: 0
> dev.em.3.mac_stats.coll_ext_errs: 0
> dev.em.3.mac_stats.xon_recvd: 0
> dev.em.3.mac_stats.xon_txd: 0
> dev.em.3.mac_stats.xoff_recvd: 0
> dev.em.3.mac_stats.xoff_txd: 0
> dev.em.3.mac_stats.total_pkts_recvd: 0
> dev.em.3.mac_stats.good_pkts_recvd: 0
> dev.em.3.mac_stats.bcast_pkts_recvd: 0
> dev.em.3.mac_stats.mcast_pkts_recvd: 0
> dev.em.3.mac_stats.rx_frames_64: 0
> dev.em.3.mac_stats.rx_frames_65_127: 0
> dev.em.3.mac_stats.rx_frames_128_255: 0
> dev.em.3.mac_stats.rx_frames_256_511: 0
> dev.em.3.mac_stats.rx_frames_512_1023: 0
> dev.em.3.mac_stats.rx_frames_1024_1522: 0
> dev.em.3.mac_stats.good_octets_recvd: 0
> dev.em.3.mac_stats.good_octets_txd: 0
> dev.em.3.mac_stats.total_pkts_txd: 0
> dev.em.3.mac_stats.good_pkts_txd: 0
> dev.em.3.mac_stats.bcast_pkts_txd: 0
> dev.em.3.mac_stats.mcast_pkts_txd: 0
> dev.em.3.mac_stats.tx_frames_64: 0
> dev.em.3.mac_stats.tx_frames_65_127: 0
> dev.em.3.mac_stats.tx_frames_128_255: 0
> dev.em.3.mac_stats.tx_frames_256_511: 0
> dev.em.3.mac_stats.tx_frames_512_1023: 0
> dev.em.3.mac_stats.tx_frames_1024_1522: 0
> dev.em.3.mac_stats.tso_txd: 0
> dev.em.3.mac_stats.tso_ctx_fail: 0
> dev.em.3.interrupts.asserts: 0
> dev.em.3.interrupts.rx_pkt_timer: 0
> dev.em.3.interrupts.rx_abs_timer: 0
> dev.em.3.interrupts.tx_pkt_timer: 0
> dev.em.3.interrupts.tx_abs_timer: 0
> dev.em.3.interrupts.tx_queue_empty: 0
> dev.em.3.interrupts.tx_queue_min_thresh: 0
> dev.em.3.interrupts.rx_desc_min_thresh: 0
> dev.em.3.interrupts.rx_overrun: 0
>
> ..............................................................................


More information about the freebsd-net mailing list