bug with some em nics on RELENG_7
Mike Tancsa
mike at sentex.net
Thu Nov 19 02:23:13 UTC 2009
At 07:29 PM 11/18/2009, Jack Vogel wrote:
>Hey Mike,
>
>Can you check if you see the same behavior on RELENG 8?
Hi Jack,
Yes, I will reboot the hardware with a RELENG_8 image tomorrow to test
>Not sure why this happens on Hartwell (82574) and not on 82571, that's
>an interesting bit, the 82574 is the ONLY interface in the em driver that
>has MSIX support, unfortunately its kinda hacked in, but it did not really
>fit into the igb driver either for various technical reasons.
Is this the "FILTER" vs "ITHREAD" ? Is there a way to force this
chipset to use the same logic as 82571s ?
# dmesg |grep ^em
em0: <Intel(R) PRO/1000 Network Connection 6.9.6> port 0xbc00-0xbc1f
mem 0xface0000-0xfacfffff,0xfacc0000-0xfacdffff irq 16 at device 0.0 on pci1
em0: Using MSI interrupt
em0: [FILTER]
em0: Ethernet address: 00:15:17:78:e6:e0
em1: <Intel(R) PRO/1000 Network Connection 6.9.6> port 0xb880-0xb89f
mem 0xfac80000-0xfac9ffff,0xfac60000-0xfac7ffff irq 17 at device 0.1 on pci1
em1: Using MSI interrupt
em1: [FILTER]
em1: Ethernet address: 00:15:17:78:e6:e1
em2: <Intel(R) PRO/1000 Network Connection 6.9.6> port 0xcc00-0xcc1f
mem 0xfade0000-0xfadfffff,0xfadc0000-0xfaddffff irq 16 at device 0.0 on pci3
em2: Using MSI interrupt
em2: [FILTER]
em2: Ethernet address: 00:15:17:cf:26:de
em3: <Intel(R) PRO/1000 Network Connection 6.9.6> port 0xc880-0xc89f
mem 0xfad80000-0xfad9ffff,0xfad60000-0xfad7ffff irq 17 at device 0.1 on pci3
em3: Using MSI interrupt
em3: [FILTER]
em3: Ethernet address: 00:15:17:cf:26:df
em4: <Intel(R) PRO/1000 Network Connection 6.9.6> port 0xdc00-0xdc1f
mem 0xfaee0000-0xfaefffff,0xfaedc000-0xfaedffff irq 16 at device 0.0 on pci6
em4: Using MSIX interrupts
em4: [ITHREAD]
em4: [ITHREAD]
em4: [ITHREAD]
em4: Ethernet address: 00:30:48:d6:ef:12
em5: <Intel(R) PRO/1000 Network Connection 6.9.6> port 0xec00-0xec1f
mem 0xfafe0000-0xfaffffff,0xfafdc000-0xfafdffff irq 17 at device 0.0 on pci7
em5: Using MSIX interrupts
em5: [ITHREAD]
em5: [ITHREAD]
em5: [ITHREAD]
em5: Ethernet address: 00:30:48:d6:ef:13
>What if you boot up, then do NOT ping or anything until the interface is
>assigned an address (and so init is run), and the cable is plugged in. If
>that happens first does it work?
yes. If I have the cables plugged in and reboot the box, its ok. I
am pretty sure all is ok if I boot it up, with no address assigned,
plug the cables in, and then assign addr.
I havent tested it out yet, but not sure how things play out when
the ports are connected to a switch that is not in portfast mode, so
the carrier does not always come up right away. The other thing I saw
was that the NIC was getting stuck with the carrier showing up, even
though cable was unplugged. However, I was not able to find the
exact conditions this happened.
>Do let me know if you can check on 8, if not I can have my validation
>engineer try this.
I will report back tomorrow.
---Mike
>Best regards,
>
>Jack
>
>
>On Wed, Nov 18, 2009 at 1:30 PM, Mike Tancsa
><<mailto:mike at sentex.net>mike at sentex.net> wrote:
>
>On two Intel chipset Supermicro boards (X8STi and X8STE-0) using the
>onboard em nics (dmesg info below), I seem to have run into an issue
>where if I boot the box up with the cables unplugged, I cannot get
>the NICS to properly work post boot up. This is quite repeatable
>for me. So at boot time, I have
>
># ifconfig em5
>em5: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
> options=19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4>
> ether 00:30:48:d6:ef:13
> inet 3.3.3.3 netmask 0xffffff00 broadcast 3.3.3.255
> media: Ethernet autoselect
> status: no carrier
>
>
>I then ping something that would be across the wire while the nic is
>down. e.g. ping 3.3.3.1
>
>I then plug in the cable so that the other side has 3.3.3.1
>
>ifconfig shows all looks good
>
># ifconfig em5
>em5: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
> options=19b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4>
> ether 00:30:48:d6:ef:13
> inet 3.3.3.3 netmask 0xffffff00 broadcast 3.3.3.255
> media: Ethernet autoselect (1000baseTX <full-duplex>)
> status: active
>
>I try and ping 3.3.3.1 which is on xover (via a switch shows the
>same behaviour), and no response to the pings.... BUT, I do see the
>MAC addr show up
># ping -c 2 -S 3.3.3.3 3.3.3.1
>PING 3.3.3.1 (3.3.3.1) from <http://3.3.3.3>3.3.3.3: 56 data bytes
>
>--- 3.3.3.1 ping statistics ---
>2 packets transmitted, 0 packets received, 100.0% packet loss
># arp -na
>? (3.3.3.1) at 00:30:48:94:88:20 on em5 [ethernet]
>? (3.3.3.3) at 00:30:48:d6:ef:13 on em5 permanent [ethernet]
>
>I can see its mac addr ?!?
>
>Furthermore, if I do
>
># ifconfig em5 <http://3.3.3.55/32>3.3.3.55/32 alias
>
>On the other side, I see
>
>0(ich10)# tcpdump -nei igb0
>tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
>listening on igb0, link-type EN10MB (Ethernet), capture size 96 bytes
>16:16:03.380886 00:30:48:d6:ef:13 > ff:ff:ff:ff:ff:ff, ethertype ARP
>(0x0806), length 60: Request who-has 3.3.3.55 tell 3.3.3.55, length 46
>
>
>and I can ping if I specify the alias as the source IP
>
># ping -S 3.3.3.55 3.3.3.1
>PING 3.3.3.1 (3.3.3.1) from <http://3.3.3.55>3.3.3.55: 56 data bytes
>64 bytes from <http://3.3.3.1>3.3.3.1: icmp_seq=0 ttl=64 time=0.184 ms
>64 bytes from <http://3.3.3.1>3.3.3.1: icmp_seq=1 ttl=64 time=0.051 ms
>64 bytes from <http://3.3.3.1>3.3.3.1: icmp_seq=2 ttl=64 time=0.055 ms
>
>
>
>16:17:01.603345 00:30:48:d6:ef:13 > 00:30:48:94:88:20, ethertype ARP
>(0x0806), length 60: Reply 3.3.3.55 is-at 00:30:48:d6:ef:13, length 46
>16:17:01.603349 00:30:48:94:88:20 > 00:30:48:d6:ef:13, ethertype
>IPv4 (0x0800), length 98: 3.3.3.1 > <http://3.3.3.55>3.3.3.55: ICMP
>echo reply, id 7946, seq 0, length 64
>16:17:02.603497 00:30:48:d6:ef:13 > 00:30:48:94:88:20, ethertype
>IPv4 (0x0800), length 98: 3.3.3.55 > <http://3.3.3.1>3.3.3.1: ICMP
>echo request, id 7946, seq 1, length 64
>16:17:02.603502 00:30:48:94:88:20 > 00:30:48:d6:ef:13, ethertype
>IPv4 (0x0800), length 98: 3.3.3.1 > <http://3.3.3.55>3.3.3.55: ICMP
>echo reply, id 7946, seq 1, length 64
>16:17:03.604510 00:30:48:d6:ef:13 > 00:30:48:94:88:20, ethertype
>IPv4 (0x0800), length 98: 3.3.3.55 > <http://3.3.3.1>3.3.3.1: ICMP
>echo request, id 7946, seq 2, length 64
>16:17:03.604516 00:30:48:94:88:20 > 00:30:48:d6:ef:13, ethertype
>IPv4 (0x0800), length 98: 3.3.3.1 > <http://3.3.3.55>3.3.3.55: ICMP
>echo reply, id 7946, seq 2, length 64
>
>
>
>but not using the initial IP addr
>
>0[iolite3A]# ping -S 3.3.3.3 3.3.3.1
>PING 3.3.3.1 (3.3.3.1) from <http://3.3.3.3>3.3.3.3: 56 data bytes
>^C
>--- 3.3.3.1 ping statistics ---
>2 packets transmitted, 0 packets received, 100.0% packet loss
>#
>
>Yet,
>
># ping -S 3.3.3.3 3.3.3.1
>PING 3.3.3.1 (3.3.3.1) from <http://3.3.3.3>3.3.3.3: 56 data bytes
>^C
>--- 3.3.3.1 ping statistics ---
>2 packets transmitted, 0 packets received, 100.0% packet loss
># ping -S 3.3.3.4 3.3.3.1
>PING 3.3.3.1 (3.3.3.1) from <http://3.3.3.4>3.3.3.4: 56 data bytes
>64 bytes from <http://3.3.3.1>3.3.3.1: icmp_seq=0 ttl=64 time=0.176 ms
>64 bytes from <http://3.3.3.1>3.3.3.1: icmp_seq=1 ttl=64 time=0.050 ms
>^C
>--- 3.3.3.1 ping statistics ---
>2 packets transmitted, 2 packets received, 0.0% packet loss
>round-trip min/avg/max/stddev = 0.050/0.113/0.176/0.063 ms
>
>
>Strange, eh ?
>
>
>em4 at pci0:6:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086
>rev=0x00 hdr=0x00
> vendor = 'Intel Corporation'
> class = network
> subclass = ethernet
> cap 01[c8] = powerspec 2 supports D0 D3 current D0
> cap 05[d0] = MSI supports 1 message, 64 bit
> cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1)
> cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled
>em5 at pci0:7:0:0: class=0x020000 card=0x10d315d9 chip=0x10d38086
>rev=0x00 hdr=0x00
> vendor = 'Intel Corporation'
> class = network
> subclass = ethernet
> cap 01[c8] = powerspec 2 supports D0 D3 current D0
> cap 05[d0] = MSI supports 1 message, 64 bit
> cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1)
> cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled
>
>
>em4: <Intel(R) PRO/1000 Network Connection 6.9.6> port 0xdc00-0xdc1f
>mem 0xfaee0000-0xfaefffff,0xfaedc000-0xfaedffff irq 16 at device 0.0 on pci6
>em4: Using MSIX interrupts
>em4: [ITHREAD]
>em4: [ITHREAD]
>em4: [ITHREAD]
>em4: Ethernet address: 00:30:48:d6:ef:12
>pcib7: <ACPI PCI-PCI bridge> irq 16 at device 28.1 on pci0
>pci7: <ACPI PCI bus> on pcib7
>em5: <Intel(R) PRO/1000 Network Connection 6.9.6> port 0xec00-0xec1f
>mem 0xfafe0000-0xfaffffff,0xfafdc000-0xfafdffff irq 17 at device 0.0 on pci7
>em5: Using MSIX interrupts
>em5: [ITHREAD]
>em5: [ITHREAD]
>em5: [ITHREAD]
>em5: Ethernet address: 00:30:48:d6:ef:13
>
>The same does NOT happen with my external 2 port pcie nics (it says
>HP, but they are intel branded)
>eg
>em0 at pci0:1:0:0: class=0x020000 card=0x115e8086 chip=0x105e8086
>rev=0x06 hdr=0x00
> vendor = 'Intel Corporation'
> device = 'HP NC360T PCIe DP Gigabit Server Adapter (n1e5132)'
> class = network
> subclass = ethernet
> cap 01[c8] = powerspec 2 supports D0 D3 current D0
> cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message
> cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x4(x4)
>em1 at pci0:1:0:1: class=0x020000 card=0x115e8086 chip=0x105e8086
>rev=0x06 hdr=0x00
> vendor = 'Intel Corporation'
> device = 'HP NC360T PCIe DP Gigabit Server Adapter (n1e5132)'
> class = network
> subclass = ethernet
> cap 01[c8] = powerspec 2 supports D0 D3 current D0
> cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message
> cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x4(x4)
>
> ---Mike
>
>--------------------------------------------------------------------
>Mike Tancsa, tel +1 519 651 3400
>Sentex
>Communications,
><mailto:mike at sentex.net>mike at sentex.net
>Providing Internet since
>1994 <http://www.sentex.net>www.sentex.net
>Cambridge, Ontario
>Canada <http://www.sentex.net/mike>www.sentex.net/mike
>
--------------------------------------------------------------------
Mike Tancsa, tel +1 519 651 3400
Sentex Communications, mike at sentex.net
Providing Internet since 1994 www.sentex.net
Cambridge, Ontario Canada www.sentex.net/mike
More information about the freebsd-stable
mailing list