[9.3 PRE] Intel i210AT ethernet adapter MSI-X problems (igb driver)

[Harald Schmalzbauer]

Bezüglich Damian Danielecki's Nachricht vom 17.05.2014 14:40 (localtime):
> I am receiving as many as 30.000-40.000 interrupts per second under no
> big nfs traffic.
> Avg traffic is 20Mb/s, avg interrupts number on igb device is 13.000/s.
> IMHO. I could compare this to similar igb adapters on other FreeBSD 9
> servers and in this case should be  less interrupts (lightweight
> traffic conditions).
> Additionally all of these interrupts occupies only one IRQ.
> The interested thing is CPU usage caused by %interrupts is very small
> (under 1%) so this is reason I am still able to use this ethernet
> card.
> 
> # uname -a
> FreeBSD nfsd.xxx.pl 9.3-PRERELEASE FreeBSD 9.3-PRERELEASE #5: Fri May
> 16 15:41:36 CEST 2014
> root at nfsd.xxx.pl:/usr/obj/usr/src/sys/FREEBSD9  amd64
> This is custom minimalist kernel.
> 
> 
> I see in the sources of igb driver that   i210 is generally supported:
> /usr/src/sys/dev/e1000 # grep 'I210' * |wc -l
>       64
> 
> I guess mine adapter (onboard quad gigabit I210T) is not correctly
> handled in the sources of the driver but system recognizes it as igb.
> 
> # dmidecode
> (...)
> Manufacturer: Supermicro
> Product Name: X10SLM+-LN4F
> (...)
> 
> # dmesg |grep igb0
> igb0: <Intel(R) PRO/1000 Network Connection version - 2.3.10> port
> 0xc000-0xc01f mem 0xf7400000-0xf747ffff,0xf7480000-0xf7483fff irq 18
> at device 0.0 on pci4
> igb0: Using MSIX interrupts with 5 vectors
> igb0: Ethernet address: 0c:c4:7a:01:e3:50
> igb0: Bound queue 0 to cpu 0
> igb0: Bound queue 1 to cpu 1
> igb0: Bound queue 2 to cpu 2
> igb0: Bound queue 3 to cpu 3
> igb0: link state changed to UP
> 
> 
> No device description:
> #pciconf -vl
> igb0 at pci0:4:0:0:        class=0x020000 card=0x153315d9 chip=0x15338086
> rev=0x03 hdr=0x00
>     vendor     = 'Intel Corporation'
>     class      = network
>     subclass   = ethernet
> 
> 
> Number of interrupts taken by device since system startup:
> # vmstat -i
> irq269: igb0:que 0             501886805      13364
> irq270: igb0:que 1                 40477          1
> irq271: igb0:que 2                 40417          1
> irq272: igb0:que 3               7526720        200
> irq273: igb0:link                     12          0
> 
> Sample current number of interrupts with not big NFS traffic:
> # systat -vm1
> Interrupts
> 34352 total (!!!)
> (...)
> 29937 igb0:que 0
> 1 igb0:que 1
> 1 igb0:que 2
> 1 igb0:que 3
> 1 igb0:link

AFAIK i210 doesn't support multiple queues.
I had to add 'hw.igb.num_queues=1' to my loader.conf when I did some
tests with i210. In my case it was PCIe-passthrough, so probably
different story, but in case you might want to see what results you get
with limiting queues to 1.

-Harry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: OpenPGP digital signature
URL: <http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20140518/eb6be249/attachment.sig>