82599 receiving packets with vlan tag=0 (vlan strip
problem)?
John Hay
jhay at meraka.org.za
Sun Nov 28 08:16:21 UTC 2010
Hi Jack,
On Sat, Nov 27, 2010 at 09:59:10AM -0800, Jack Vogel wrote:
> Well, that will be cool if so, its not usually FreeBSD that
> finds bugs, and if it really is hardware then they need to
> know. Thanks Ryan!
>
> Jack
>
>
> On Sat, Nov 27, 2010 at 4:52 AM, Ryan Stone <rysto32 at gmail.com> wrote:
>
> > 2010/11/26 Jack Vogel <jfvogel at gmail.com>:
> > > Just for the record, I was not aware of a hardware bug, and
> > > for right now no one is around :) I will ask around next week
> > > to see if something was known that I missed.
> >
> > I doubt that anybody at Intel will know about this one. To my
> > knowledge, it's not in the errata for the 82599. This is just
> > something that I discovered independently over the summer, fixed in my
> > local tree and promptly forgot about.
I don't think your solution / workaround work in every case. To debug my
other vlan + ipv6 problem, I have configured a mirrored port on the switch
and connected that to one of the ixgbe interfaces with no vlans configured
on it. Packets dumped on it looks like this:
#########################################
# tcpdump -i ix3 -n -s 0 -e ether host 00:23:ae:a5:00:ef
tcpdump: WARNING: ix3: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ix3, link-type EN10MB (Ethernet), capture size 65535 bytes
10:00:58.215870 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 8, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 0, length 16
10:00:59.207090 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 8, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 1, length 16
10:01:00.206506 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 8, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 2, length 16
10:01:01.206923 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 8, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 3, length 16
10:01:02.206378 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 8, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 4, length 16
10:01:03.168219 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 86: vlan 0, p 0, ethertype 802.1Q, vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, neighbor advertisement, tgt is fe80::21b:21ff:fe57:b420, length 24
10:01:03.206846 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 5, length 16
10:01:03.215452 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 94: vlan 0, p 0, ethertype 802.1Q, vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, neighbor solicitation, who has fe80::223:aeff:fea5:ef, length 32
10:01:04.206331 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 6, length 16
10:01:05.206760 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 7, length 16
10:01:06.206203 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 8, length 16
10:01:07.206794 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 8, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 9, length 16
########################################
On the base interface that actually sent the packets out, it looked like
this:
########################################
# tcpdump -i ix2 -n -s 0 -e ether host 00:23:ae:a5:00:ef
tcpdump: WARNING: ix2: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ix2, link-type EN10MB (Ethernet), capture size 65535 bytes
10:00:58.215838 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 0, length 16
10:00:58.215861 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 0, length 16
10:00:59.207067 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 1, length 16
10:00:59.207081 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 1, length 16
10:01:00.206484 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 2, length 16
10:01:00.206497 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 2, length 16
10:01:01.206899 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 3, length 16
10:01:01.206913 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 3, length 16
10:01:02.206355 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 4, length 16
10:01:02.206368 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 4, length 16
10:01:03.168187 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q (0x8100), length 90: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > fe80::21b:21ff:fe57:b420: ICMP6, neighbor solicitation, who has fe80::21b:21ff:fe57:b420, length 32
10:01:03.168210 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 82: vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, neighbor advertisement, tgt is fe80::21b:21ff:fe57:b420, length 24
10:01:03.206828 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 5, length 16
10:01:03.206838 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 5, length 16
10:01:03.215445 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 90: vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, neighbor solicitation, who has fe80::223:aeff:fea5:ef, length 32
10:01:03.215604 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q (0x8100), length 82: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > fe80::21b:21ff:fe57:b420: ICMP6, neighbor advertisement, tgt is fe80::223:aeff:fea5:ef, length 24
10:01:04.206307 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 6, length 16
10:01:04.206321 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 6, length 16
10:01:05.206739 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 7, length 16
10:01:05.206751 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 7, length 16
10:01:06.206181 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 8, length 16
10:01:06.206194 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 8, length 16
10:01:07.206772 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 9, length 16
########################################
You can see that there is only one vlan tag.
You can also see that all of these were sent out on vlan 1, while by the
time they arrive on the mirrored port some have been mangled to vlan 8.
This is after updating to the new driver that was merged today. It seems
that I do not need routing anymore for this condition to happen. Just a
ping to the link-local address triggers it.
I did this from another machine:
########################################
# ping6 fe80::21b:21ff:fe57:b420%em0
PING6(56=40+8+8 bytes) fe80::223:aeff:fea5:ef%em0 --> fe80::21b:21ff:fe57:b420%em0
16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=5 hlim=64 time=0.180 ms
16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=6 hlim=64 time=0.134 ms
16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=7 hlim=64 time=0.145 ms
16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=8 hlim=64 time=0.169 ms
16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=33 hlim=64 time=0.135 ms
16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=34 hlim=64 time=0.145 ms
16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=35 hlim=64 time=0.151 ms
16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=36 hlim=64 time=0.166 ms
16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=37 hlim=64 time=0.159 ms
16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=38 hlim=64 time=0.147 ms
16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=39 hlim=64 time=0.146 ms
16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=40 hlim=64 time=0.213 ms
^C
--- fe80::21b:21ff:fe57:b420%em0 ping6 statistics ---
41 packets transmitted, 12 packets received, 70.7% packet loss
round-trip min/avg/max/std-dev = 0.134/0.158/0.213/0.021 ms
########################################
Currently the configs look like this:
########################################
mr3# ifconfig ix2
ix2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=1b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4>
ether 00:1b:21:57:ef:7c
inet6 fe80::21b:21ff:fe57:ef7c%ix2 prefixlen 64 scopeid 0x3
nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
media: Ethernet autoselect (10Gbase-SR <full-duplex>)
status: active
mr3# ifconfig ix3
ix3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=1bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4>
ether 00:1b:21:57:ef:7d
inet6 fe80::21b:21ff:fe57:ef7d%ix3 prefixlen 64 scopeid 0x4
nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
media: Ethernet autoselect (10Gbase-SR <full-duplex>)
status: active
mr3# ifconfig ix2.1
ix2.1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 00:1b:21:57:ef:7c
inet 146.64.28.2 netmask 0xffffff00 broadcast 146.64.28.255
inet6 fe80::21b:21ff:fe57:b420%ix2.1 prefixlen 64 scopeid 0xa
inet6 2001:4200:7000:3:21b:21ff:fe57:b420 prefixlen 64
inet6 2001:4200:7000:3:: prefixlen 64 anycast
nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
media: Ethernet autoselect (10Gbase-SR <full-duplex>)
status: active
vlan: 1 parent interface: ix2
mr3# ifconfig ix2.8
ix2.8: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 00:1b:21:57:ef:7c
inet 146.64.8.50 netmask 0xffffff00 broadcast 146.64.8.255
inet6 fe80::21b:21ff:fe57:b420%ix2.8 prefixlen 64 scopeid 0xb
inet6 2001:4200:7000:1:21b:21ff:fe57:b420 prefixlen 64
inet6 2001:4200:7000:1:: prefixlen 64 anycast
nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
media: Ethernet autoselect (10Gbase-SR <full-duplex>)
status: active
vlan: 8 parent interface: ix2
########################################
John
--
John Hay -- jhay at meraka.csir.co.za / jhay at FreeBSD.org
More information about the freebsd-net
mailing list