Dropped vs. missed packets in the ixgbe driver

[Barney Cordoba]

--- On Tue, 8/18/09, Manish Vachharajani <manishv at lineratesystems.com> wrote:

> From: Manish Vachharajani <manishv at lineratesystems.com>
> Subject: Re: Dropped vs. missed packets in the ixgbe driver
> To: "Barney Cordoba" <barney_cordoba at yahoo.com>
> Cc: freebsd-net at freebsd.org
> Date: Tuesday, August 18, 2009, 6:35 PM
> Indeed the debugging info is also
> interesting.  However, I'd like to
> get some data from netstat when the driver drops
> frames.  When looking
> at the bge driver source, it appears that all input drops
> at the NIC
> are reported as input errors.  It appears that the
> intel drivers
> (e1000, ixgb, and ixgbe) don't report that number outside
> of the debug
> and stats printfs at all, and this seems broken.  What
> I want to know
> is if I have just missed where these are reported.
> 
> So, in a nutshell, the question is:  should these
> drivers be reporting
> miss events as input errors in the ifnet struct as the bge
> driver
> does, or as drops in the ifnet struct, was there some
> conscious
> decision not to report miss events anywhere outside the
> debug and
> stats info, or am I just being silly and not seeing where
> the numbers
> are reported?
> 
> Also, don't worry on the performance front, we are also
> looking at the
> driver in FreeBSD 8.0 :) which supports RSS to help
> performance
> scaling,  though we have some interesting data there
> that I'll post
> about once I confirm that the numbers are indeed correct
> and not a
> tuning or setup problem.
> 
> Manish
> 
> > --- On Mon, 8/17/09, Manish Vachharajani <manishv at lineratesystems.com>
> wrote:
> >
> >> From: Manish Vachharajani <manishv at lineratesystems.com>
> >> Subject: Dropped vs. missed packets in the ixgbe
> driver
> >> To: freebsd-net at freebsd.org
> >> Date: Monday, August 17, 2009, 6:24 PM
> >> I've been doing some performance
> >> testing on freebsd 7.2 and noticed
> >> that the ixgbe driver does not report missed
> packets as
> >> dropped when
> >> queried via netstat -id (the ixgbe driver in the
> 8.0
> >> beta has a
> >> similar issue).  A missed packet is a packet that
> was
> >> correctly
> >> received by the NIC but because it was out of
> descriptors
> >> and internal
> >> memory, the packet had to be dropped by the NIC
> itself -- a
> >> hardware
> >> register counts such events.  Instead the driver
> only
> >> reports drops
> >> that are due to the driver itself, i.e., the NIC
> DMAed the
> >> packet to
> >> memory but the driver had to drop something
> because it was
> >> out of
> >> mbufs.  Is the miss count reported elsewhere
> (besides
> >> via a kernel
> >> printf from the driver)?  At the end of the email
> I
> >> give a stats dump
> >> from the driver and from netstat, note the number
> of
> >> packets dropped
> >> is 0 as reported by netstat but the Missed field
> in the
> >> dmesg output
> >> shows many missed packets.
> >>
> >> >From my perspective, it is disconcerting to
> see
> >> performance
> >> degradation on the link, along with TCP ack
> retransmits,
> >> packet
> >> reordering, etc. (on a point-to-point link with no
> switch
> >> in between)
> >> but then see no drops reported by netstat because
> the
> >> driver didn't
> >> drop the packet, the NIC did.  The fix should be
> >> straight-forward and
> >> I'll gladly make a patch assuming that it is
> indeed a bug
> >> and not a
> >> conscious design choice.
> >>
> >> Here is the relevant netstat output
> >>
> >> Name    Mtu Network
> >>    Address
> >>     Ipkts Ierrs    Opkts
> >> Oerrs  Coll Drop
> >> ix0    1500 <Link#1>
> >> 00:30:48:94:60:ec        0
> >>    0        1
> >>  0     0    0
> >> ix0    1500 192.168.105.0 192.168.105.2
> >>           0
> >>    -        0
> >>  -     -    -


if you look in ixgbe_update_stats_counters at the bottom:

        ifp->if_ierrors = missed_rx + adapter->stats.crcerrs +
                adapter->stats.rlec;

the errors are added in. 

BC
> >> 00:30:48:94:60:ed      11M
> >>    0     6.1M
> >>  0     0    0
> >> ix1    1500
> >> 192.168.5.0   192.168.5.2
> >>       10M     -
> >>    6.1M
> >>  -     -    -
> >>
> >> And here is the dmesg output after doing a sysctl
> >> dev.ix.1.stats=1
> >>
> >> ix1: Std Mbuf Failed = 0
> >> ix1: Missed Packets = 413872
> >> ix1: Receive length errors = 0
> >> ix1: Crc errors = 0
> >> ix1: Driver dropped packets = 0
> >> ix1: watchdog timeouts = 0
> >> ix1: XON Rcvd = 616428212235
> >> ix1: XON Xmtd = 0
> >> ix1: XOFF Rcvd = 616428212235
> >> ix1: XOFF Xmtd = 0
> >> ix1: Total Packets Rcvd = 12424533
> >> ix1: Good Packets Rcvd = 12010661
> >> ix1: Good Packets Xmtd = 6419128
> >> ix1: TSO Transmissions = 0
> >>
> >> Manish
> >
> > the debug sysctl show more interesting info. Don't get
> too excited
> > about doing performance testing with that driver. Its
> not designed
> > to be any higher in performance than any of the other
> intel drivers.
> >
> > Barney
> >
> >
> >
> >
> 
> 
> 
> -- 
> Manish Vachharajani
> Founder
> LineRate Systems
> manishv at lineratesystems.com
> (609)635-9531 M
> _______________________________________________
> freebsd-net at freebsd.org
> mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>