High interrupt rate on a PF box + performance

[Vogel, Jack]

Jeremy is right, if you have a problem the first step is to try the latest code.

However, when I look at the interrupts below I don't see what the problem is?
The Broadcom seems to have about the same rate, it just doesn't have MSIX (multiple vectors).

Jack


-----Original Message-----
From: Damien Fleuriot [mailto:ml at my.gd] 
Sent: Thursday, January 27, 2011 9:31 AM
To: Jeremy Chadwick
Cc: freebsd-stable at freebsd.org; freebsd-pf at freebsd.org; Vogel, Jack
Subject: Re: High interrupt rate on a PF box + performance



On 1/27/11 6:27 PM, Jeremy Chadwick wrote:
> On Thu, Jan 27, 2011 at 10:57:14AM +0100, Damien Fleuriot wrote:
>> Hello list,
>>
>> I have a problem with interrupts, network cards, and PF performance.
>>
>> We have 2 firewalls running FreeBSD 8.0 for the current master and
>> FreeBSD 8.1 for the backup host, which I upgraded just yesterday.
>>
>> [...]
>>
>> vmstat -i
>> ---
>> interrupt                          total       rate
>> irq14: ata0                           36          0
>> irq16: mfi0                       353244          1
>> irq21: uhci0 uhci+                461504          1
>> cpu0: timer                    615183815       1996
>> irq256: bce0                  1015412475       3295
>> irq257: igb0                  1067318584       3464
>> irq258: igb0                   695648752       2258
>> irq259: igb0                           2          0
>> irq260: igb1                    11503857         37
>> irq261: igb1                      506598          1
>> irq262: igb1                          69          0
>> irq269: bce1                      790820          2
>> cpu1: timer                    615183757       1996
>> cpu2: timer                    615197165       1996
>> cpu3: timer                    615197165       1996
>> Total                         5252757843      17050
> 
> There are changes to the igb(4) driver which are in RELENG_8 (8-STABLE),
> and some which will be in the upcoming 8.2-RELEASE, which may address
> this.  Jack Vogel of Intel would be able to confirm for sure; CC'ing him
> here.
> 
> Could you please provide output from the following commands?
> 
> * pciconf -lvcb  (only include igbX entries, thanks)
> * sysctl -a | grep msi
> 
> Thanks.
> 
> I can't help with the CARP-related issues or other stuff you're
> experiencing.  These issues may all be separate problems, hard to say.
> 


igb0 at pci0:14:0:0:	class=0x020000 card=0x145a8086 chip=0x10d68086
rev=0x02 hdr=0x00
igb1 at pci0:14:0:1:	class=0x020000 card=0x145a8086 chip=0x10d68086
rev=0x02 hdr=0x00
igb2 at pci0:15:0:0:	class=0x020000 card=0x145a8086 chip=0x10d68086
rev=0x02 hdr=0x00
igb3 at pci0:15:0:1:	class=0x020000 card=0x145a8086 chip=0x10d68086
rev=0x02 hdr=0x00

hw.bce.msi_enable: 1
hw.pci.honor_msi_blacklist: 1
hw.pci.enable_msix: 1
hw.pci.enable_msi: 1


igb0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric
0 mtu 1500
	options=13b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,TSO4>
	ether 00:1b:21:12:ec:38
	inet [snip] netmask 0xffffffc0 broadcast [snip]
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active

igb1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=13b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,TSO4>
	ether 00:1b:21:12:ec:39
	inet 10.0.0.252 netmask 0xffffff00 broadcast 10.0.0.255
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active


Here you go :)

Note that the igb2 and 3 interfaces are unused, unplugged, unconfigured