High CPU interrupt load on intel I350T4 with igb on 8.3

[Barney Cordoba]

--- On Sat, 5/11/13, Hooman Fazaeli <hoomanfazaeli at gmail.com> wrote:

> From: Hooman Fazaeli <hoomanfazaeli at gmail.com>
> Subject: Re: High CPU interrupt load on intel I350T4 with igb on 8.3
> To: "Barney Cordoba" <barney_cordoba at yahoo.com>
> Cc: "Eugene Grosbein" <egrosbein at rdtc.ru>, freebsd-net at freebsd.org, ""Clément Hermann (nodens)"" <nodens2099 at gmail.com>
> Date: Saturday, May 11, 2013, 4:12 PM
> On 5/11/2013 8:26 PM, Barney Cordoba
> wrote:
> > Clearly you don't understand the problem. Your logic is
> that because other drivers are defective also; therefore its
> not a driver problem? The problem is caused by a
> multi-threaded driver that
> > haphazardly launches tasks and that doesn't manage the
> case that the rest of the system can't handle the load. It's
> no different than a driver that barfs when mbuf clusters are
> exhausted. The answer
> > isn't to increase memory or mbufs, even though that may
> alleviate the problem. The answer is to fix the driver, so
> that it doesn't crash the system for an event that is wholly
> predictable. igb has
> > 1) too many locks and 2) exasperates the problem by
> binding to cpus, which causes it to not only have to wait
> for the lock to free, but also for a specific cpu to become
> free. So it chugs along
> > happily until it encounters a bottleneck, at which
> point it quickly blows up the entire system in a domino
> effect. It needs to manage locks more efficiently, and also
> to detect when the backup is
> > unmanageable. Ever since FreeBSD 5 the answer has been
> "it's fixed in 7, or its fixed in 9, or it's fixed in 10".
> There will always be bottlenecks, and no driver should blow
> up the system no matter
> > what intermediate code may present a problem. Its the
> driver's responsibility to behave and to drop packets if
> necessary. BC
> 
> And how the driver should behave? You suggest dropping the
> packets. Even if we accept
> that dropping packets is a good strategy in all
> configurations (which I doubt), the driver is
> definitely not the best place to implement it, since that
> involves duplication of similar
> code between drivers. Somewhere like the Ethernet layer is a
> much better choice to watch
> load of packets and drop them to prevent them to eat all the
> cores. Furthermore, ignoring
> the fact that pf is not optimized for multi-processors and
> blaming drivers for not adjusting
> themselves with the this pf's fault, is a bit unfair, I
> believe.
> 

It's easier to make excuses than to write a really good driver. I'll
grant you that.
BC