High CPU interrupt load on intel I350T4 with igb on 8.3

[Eugene Grosbein]

On 26.04.2013 18:31, "Clément Hermann (nodens)" wrote:
> Hi list,
> 
> We use pf+ALTQ for trafic shaping on some routers.
> 
> We are switching to new servers : Dell PowerEdge R620 with 2 8-cores 
> Intel Processor (E5-2650L), 8GB RAM and Intel I350T4 (quad port) using 
> igb driver. The old hardware is using em driver, the CPU load is high 
> but mostly due to kernel and a large pf ruleset.
> 
> On the new hardware, we see high CPU Interrupt load (up to 95%), even 
> though there is not much trafic currently (peaks about 150Mbps and 
> 40Kpps). All queues are used and binded to a cpu according to top, but a 
> lot of CPU time is spent on igb queues (interrupt or wait). The load is 
> fine when we stay below 20Kpps.
> 
> We see no mbuf shortage, no dropped packet, but there is little margin 
> left on CPU time (about 25% idle at best, most of CPU time is spent on 
> interrupts), which is disturbing.

It seems you suffer from pf lock contention. You should stop using pf
with multi-core systems with 8.3. Move to ipfw+dummynet or ng_car for 8.3
or move to 10.0-CURRENT having new, rewritten pf that does not have this problem.

Network device driver is not guilty here, that's just pf's contention
running in igb's context.

Eugene Grosbein