Intel 82574L Gigabit Ethernet Controller
Jack Vogel
jfvogel at gmail.com
Wed Jul 7 00:09:10 UTC 2010
Yow, 30 vlans, but only em1 is using vlans not em0?
Is only em1 having watchdogs? I noticed you appear to
have flow control off, maybe turning it on would help.
I would like to see the log messages from the watchdogs.
Jack
On Tue, Jul 6, 2010 at 1:28 AM, Shtorm <admin at shtorm.com> wrote:
> On Mon, 2010-07-05 at 22:02 -0700, Jack Vogel wrote:
> > Cleaner in the Makefile, let me know how it goes.
> >
> > Jack
> >
> >
> > On Mon, Jul 5, 2010 at 12:06 PM, Yuriy A. Korobko
> > <administrator at shtorm.com>wrote:
> >
> > > On Mon, 2010-07-05 at 11:11 -0700, Jack Vogel wrote:
> > > > Are you defining 'EM_MULTIQUEUE', its off by default and needs to be
> > > > defined somewhere by you.
> > > >
> > > > You will only see the two queues used if you have two different
> > > connections
> > > > operating at once.
> > > >
> > > > Jack
>
> Recompiled module (actually whole kernel) with EM_MULTIQUEUE, but got a
> some watchdog timeouts.
>
> Here is example of netstat when it happens:
>
> 2346 0 0 583370 2642 0 2988379 0
> 2250 0 0 550961 2384 0 2835276 0
> 2634 0 0 703410 2733 0 2971417 0
> 2741 0 0 695884 2748 0 3061573 0
> 2811 0 0 618520 2274 0 3273868 0
> 3220 0 0 687145 2068 0 3417857 0
> 901 0 0 231694 166 0 1565839 0
> 752 0 0 216135 0 0 270247 0
> 685 0 0 208745 0 0 243442 0
> 713 0 0 178089 0 0 230472 0
> 735 0 0 159555 0 0 178435 0
> 616 0 0 145222 0 0 179022 0
> input (em1) output
> packets errs idrops bytes packets errs bytes colls
> 616 0 0 129608 0 0 120929 0
> 659 0 0 113806 0 0 105707 0
> 622 0 0 106247 0 0 107825 0
> 645 0 0 101593 0 0 38023 0
> 483 0 0 61681 0 0 8547 0
>
> Watchdog timeout on em1 message on console
>
> 0 0 0 0 0 1 19095 0
> 0 0 0 0 0 0 222 0
> 0 0 0 0 0 0 0 0
> 243 0 0 30224 1783 0 1708875 0
> 576 0 0 62350 105 0 7710 0
> 467 0 0 49758 4 0 4388 0
> 500 0 0 53415 23 0 24743 0
> 437 0 0 50135 21 0 16733 0
>
> After this traffic increased again up to 3 kpps and another watchdog
> timeout happened.
>
> em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu
> 1500
> options=2098<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC>
> ether 00:30:48:bc:ab:ca
> inet x.x.x.x netmask 0xffffffe0 broadcast x.x.x.y
> media: Ethernet autoselect (1000baseT <full-duplex>)
> status: active
>
> em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu
> 1500
> options=2098<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC>
> ether 00:30:48:bc:ab:cb
> media: Ethernet autoselect (1000baseT <full-duplex>)
> status: active
>
> em1 have 30 vlans and mpd5 as pppoe server listening on each vlan.
>
> Here is sysctl dev.em
>
> dev.em.0.%desc: Intel(R) PRO/1000 Network Connection 7.0.5
> dev.em.0.%driver: em
> dev.em.0.%location: slot=0 function=0
> dev.em.0.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9
> subdevice=0x10d3 class=0x020000
> dev.em.0.%parent: pci1
> dev.em.0.nvm: -1
> dev.em.0.rx_int_delay: 200
> dev.em.0.tx_int_delay: 200
> dev.em.0.rx_abs_int_delay: 500
> dev.em.0.tx_abs_int_delay: 500
> dev.em.0.rx_processing_limit: 100
> dev.em.0.link_irq: 2
> dev.em.0.mbuf_alloc_fail: 0
> dev.em.0.cluster_alloc_fail: 0
> dev.em.0.dropped: 0
> dev.em.0.tx_dma_fail: 0
> dev.em.0.fc_high_water: 18432
> dev.em.0.fc_low_water: 16932
> dev.em.0.mac_stats.excess_coll: 0
> dev.em.0.mac_stats.symbol_errors: 0
> dev.em.0.mac_stats.sequence_errors: 0
> dev.em.0.mac_stats.defer_count: 0
> dev.em.0.mac_stats.missed_packets: 0
> dev.em.0.mac_stats.recv_no_buff: 0
> dev.em.0.mac_stats.recv_errs: 0
> dev.em.0.mac_stats.crc_errs: 0
> dev.em.0.mac_stats.alignment_errs: 0
> dev.em.0.mac_stats.coll_ext_errs: 0
> dev.em.0.mac_stats.rx_overruns: 0
> dev.em.0.mac_stats.watchdog_timeouts: 0
> dev.em.0.mac_stats.xon_recvd: 0
> dev.em.0.mac_stats.xon_txd: 0
> dev.em.0.mac_stats.xoff_recvd: 0
> dev.em.0.mac_stats.xoff_txd: 0
> dev.em.0.mac_stats.total_pkts_recvd: 732916
> dev.em.0.mac_stats.good_pkts_recvd: 732916
> dev.em.0.mac_stats.bcast_pkts_recvd: 1238
> dev.em.0.mac_stats.mcast_pkts_recvd: 0
> dev.em.0.mac_stats.rx_frames_64: 84622
> dev.em.0.mac_stats.rx_frames_65_127: 128831
> dev.em.0.mac_stats.rx_frames_128_255: 34037
> dev.em.0.mac_stats.rx_frames_256_511: 30206
> dev.em.0.mac_stats.rx_frames_512_1023: 24919
> dev.em.0.mac_stats.rx_frames_1024_1522: 430301
> dev.em.0.mac_stats.good_octets_recvd: 0
> dev.em.0.mac_stats.good_octest_txd: 0
> dev.em.0.mac_stats.total_pkts_txd: 678078
> dev.em.0.mac_stats.good_pkts_txd: 678078
> dev.em.0.mac_stats.bcast_pkts_txd: 109
> dev.em.0.mac_stats.mcast_pkts_txd: 0
> dev.em.0.mac_stats.tx_frames_64: 324803
> dev.em.0.mac_stats.tx_frames_65_127: 196866
> dev.em.0.mac_stats.tx_frames_128_255: 47362
> dev.em.0.mac_stats.tx_frames_256_511: 31917
> dev.em.0.mac_stats.tx_frames_512_1023: 44997
> dev.em.0.mac_stats.tx_frames_1024_1522: 32133
> dev.em.0.mac_stats.tso_txd: 0
> dev.em.0.mac_stats.tso_ctx_fail: 0
> dev.em.0.interrupts.asserts: 0
> dev.em.0.interrupts.rx_pkt_timer: 0
> dev.em.0.interrupts.rx_abs_timer: 0
> dev.em.0.interrupts.tx_pkt_timer: 0
> dev.em.0.interrupts.tx_abs_timer: 0
> dev.em.0.interrupts.tx_queue_empty: 0
> dev.em.0.interrupts.tx_queue_min_thresh: 0
> dev.em.0.interrupts.rx_desc_min_thresh: 0
> dev.em.0.interrupts.rx_overrun: 0
> dev.em.0.host.breaker_tx_pkt: 0
> dev.em.0.host.host_tx_pkt_discard: 0
> dev.em.0.host.rx_pkt: 0
> dev.em.0.host.breaker_rx_pkts: 0
> dev.em.0.host.breaker_rx_pkt_drop: 0
> dev.em.0.host.tx_good_pkt: 0
> dev.em.0.host.breaker_tx_pkt_drop: 0
> dev.em.0.host.rx_good_bytes: 0
> dev.em.0.host.tx_good_bytes: 0
> dev.em.0.host.length_errors: 0
> dev.em.0.host.serdes_violation_pkt: 0
> dev.em.0.host.header_redir_missed: 0
> dev.em.1.%desc: Intel(R) PRO/1000 Network Connection 7.0.5
> dev.em.1.%driver: em
> dev.em.1.%location: slot=0 function=0
> dev.em.1.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x15d9
> subdevice=0x10d3 class=0x020000
> dev.em.1.%parent: pci2
> dev.em.1.nvm: -1
> dev.em.1.rx_int_delay: 200
> dev.em.1.tx_int_delay: 200
> dev.em.1.rx_abs_int_delay: 500
> dev.em.1.tx_abs_int_delay: 500
> dev.em.1.rx_processing_limit: 100
> dev.em.1.link_irq: 7
> dev.em.1.mbuf_alloc_fail: 0
> dev.em.1.cluster_alloc_fail: 0
> dev.em.1.dropped: 0
> dev.em.1.tx_dma_fail: 0
> dev.em.1.fc_high_water: 18432
> dev.em.1.fc_low_water: 16932
> dev.em.1.mac_stats.excess_coll: 0
> dev.em.1.mac_stats.symbol_errors: 0
> dev.em.1.mac_stats.sequence_errors: 0
> dev.em.1.mac_stats.defer_count: 0
> dev.em.1.mac_stats.missed_packets: 0
> dev.em.1.mac_stats.recv_no_buff: 0
> dev.em.1.mac_stats.recv_errs: 0
> dev.em.1.mac_stats.crc_errs: 0
> dev.em.1.mac_stats.alignment_errs: 0
> dev.em.1.mac_stats.coll_ext_errs: 0
> dev.em.1.mac_stats.rx_overruns: 0
> dev.em.1.mac_stats.watchdog_timeouts: 3
> dev.em.1.mac_stats.xon_recvd: 0
> dev.em.1.mac_stats.xon_txd: 0
> dev.em.1.mac_stats.xoff_recvd: 0
> dev.em.1.mac_stats.xoff_txd: 0
> dev.em.1.mac_stats.total_pkts_recvd: 638131
> dev.em.1.mac_stats.good_pkts_recvd: 638131
> dev.em.1.mac_stats.bcast_pkts_recvd: 49276
> dev.em.1.mac_stats.mcast_pkts_recvd: 0
> dev.em.1.mac_stats.rx_frames_64: 8091
> dev.em.1.mac_stats.rx_frames_65_127: 472864
> dev.em.1.mac_stats.rx_frames_128_255: 46165
> dev.em.1.mac_stats.rx_frames_256_511: 33998
> dev.em.1.mac_stats.rx_frames_512_1023: 44560
> dev.em.1.mac_stats.rx_frames_1024_1522: 32453
> dev.em.1.mac_stats.good_octets_recvd: 0
> dev.em.1.mac_stats.good_octest_txd: 0
> dev.em.1.mac_stats.total_pkts_txd: 668981
> dev.em.1.mac_stats.good_pkts_txd: 668981
> dev.em.1.mac_stats.bcast_pkts_txd: 0
> dev.em.1.mac_stats.mcast_pkts_txd: 12
> dev.em.1.mac_stats.tx_frames_64: 5594
> dev.em.1.mac_stats.tx_frames_65_127: 166028
> dev.em.1.mac_stats.tx_frames_128_255: 33061
> dev.em.1.mac_stats.tx_frames_256_511: 28853
> dev.em.1.mac_stats.tx_frames_512_1023: 24279
> dev.em.1.mac_stats.tx_frames_1024_1522: 411166
> dev.em.1.mac_stats.tso_txd: 0
> dev.em.1.mac_stats.tso_ctx_fail: 0
> dev.em.1.interrupts.asserts: 0
> dev.em.1.interrupts.rx_pkt_timer: 0
> dev.em.1.interrupts.rx_abs_timer: 0
> dev.em.1.interrupts.tx_pkt_timer: 0
> dev.em.1.interrupts.tx_abs_timer: 0
> dev.em.1.interrupts.tx_queue_empty: 0
> dev.em.1.interrupts.tx_queue_min_thresh: 0
> dev.em.1.interrupts.rx_desc_min_thresh: 0
> dev.em.1.interrupts.rx_overrun: 0
> dev.em.1.host.breaker_tx_pkt: 0
> dev.em.1.host.host_tx_pkt_discard: 0
> dev.em.1.host.rx_pkt: 0
> dev.em.1.host.breaker_rx_pkts: 0
> dev.em.1.host.breaker_rx_pkt_drop: 0
> dev.em.1.host.tx_good_pkt: 0
> dev.em.1.host.breaker_tx_pkt_drop: 0
> dev.em.1.host.rx_good_bytes: 0
> dev.em.1.host.tx_good_bytes: 0
> dev.em.1.host.length_errors: 0
> dev.em.1.host.serdes_violation_pkt: 0
> dev.em.1.host.header_redir_missed: 0
>
> loader.conf variables:
>
> if_em_load=YES
> hw.em.rxd=4096
> hw.em.txd=4096
> hw.em.tx_int_delay=200
> hw.em.rx_int_delay=200
> hw.em.tx_abs_int_delay=500
> hw.em.rx_abs_int_delay=500
> hw.em.enable_msix=1
> hw.em.msix_queues=2
> hw.em.rx_process_limit=100
> hw.em.fc_setting=0
>
>
> And here is vmstat -i
>
> interrupt total rate
> irq16: uhci0 2673 2
> irq18: ehci0 uhci5 2 0
> irq19: uhci2 uhci4 70 0
> irq23: uhci3 ehci1 8835 9
> cpu0: timer 3743479 3957
> irq256: em0 662219 700
> irq257: em0 717531 758
> irq259: em0 12142 12
> irq260: em0 6100 6
> irq261: em1 694169 733
> irq262: em1 217388 229
> irq264: em1 463023 489
> irq265: em1 11 0
> cpu1: timer 3743411 3957
> cpu6: timer 3743408 3957
> cpu7: timer 3743407 3957
> cpu4: timer 3743408 3957
> cpu5: timer 3743407 3957
> cpu3: timer 3743408 3957
> cpu2: timer 3743409 3957
> Total 32731500 34599
>
>
> If you need more information just let me know, I can use this box for
> tests at night until others can handle traffic.
>
>
>
More information about the freebsd-net
mailing list