aliases on em(4) do not work properly
Josh Carroll
josh.carroll at gmail.com
Thu May 19 22:36:31 UTC 2011
After upgrading my hardware, I now have two new em(4) in this box
running FreeBSD 8.2-RELEASE/amd64. One NIC is the onboard NIC on the
Asus P8Z68-V Pro board, the other is the Intel EXPI9301CTBLK
PCI-Express card. em0 is the internet-facing card and I have 4 aliases
in addition to the primary address (I have a /29 IP allocation from
Comcast Business Class). After this hardware upgrade, any connection
with a source address of one of the aliases on em0 does not work
properly, it just hangs. Here is my rc.conf entry for this interface:
ipv4_addrs_em0="XYZ.14.73.185-189/29"
I also tried with the deprecated way (ifconfig_em0_alias0="..."). So
connections using the default IP (XYZ.14.73.185) work fine, for
example I can do the following:
ping -S XYZ.14.73.185 www.google.com
But if I try:
ping -S XYZ.14.73.186 www.google.com
It drops all packets. If I modify rc.conf to use .186 as the primary
IP address and make .185 an alias, then I am able to use .186 to
initiate connections. What's even more interesting is I can then set
the primary IP back to .185 and have .186 as an alias again, and I can
still use that IP. I thought it might be a strange arp problem, but
even after putting .185 back as the default IP, and doing an arp -a -d
to wipe the arp table, the .186 alias still works. I haven't rebooted,
but I suspect as soon as I reboot the problem will reoccur. I can
confirm that later this evening when I'm physically with the hardware.
I saw a thread a couple of years ago with a similar issue:
http://lists.freebsd.org/pipermail/freebsd-stable/2009-January/047839.html
So I don't know if this is the same problem or just the same symptoms.
Here is a tcpdump on em0 (tcpdump -n -vvv -i em0 host XYZ.14.73.188)
during a ping -S XYZ.14.73.188 (I had to use .188 because .186 and
.187 work now after making them the default IP as I mentioned above):
13:52:00.794099 IP (tos 0x0, ttl 64, id 20808, offset 0, flags [none],
proto ICMP (1), length 84)
XYZ.14.73.188 > 74.125.224.114: ICMP echo request, id 14907, seq
0, length 64
13:52:00.817650 IP (tos 0x20, ttl 53, id 43491, offset 0, flags
[none], proto ICMP (1), length 84)
74.125.224.114 > XYZ.14.73.188: ICMP echo reply, id 14907, seq 0, length 64
So traffic seems to pass and responses come back, but I cannot
complete a TCP handshake. Here's a tcpdump of an attempt to telnet to
www.google.com on port 80:
13:54:12.662323 IP (tos 0x20, ttl 53, id 61451, offset 0, flags
[none], proto TCP (6), length 60)
74.125.224.113.80 > XYZ.14.73.188.21613: Flags [S.], cksum 0x6796
(correct), seq 1796974652, ack 1835043040, win 5672, options [mss
1430,sackOK,TS val 3210048141 ecr 14620543,nop,wscale 6], length 0
13:54:13.049625 IP (tos 0x20, ttl 53, id 61451, offset 0, flags
[none], proto TCP (6), length 60)
74.125.224.113.80 > XYZ.14.73.188.21613: Flags [S.], cksum 0x6613
(correct), seq 1796974652, ack 1835043040, win 5672, options [mss
1430,sackOK,TS val 3210048528 ecr 14620543,nop,wscale 6], length 0
13:54:13.650624 IP (tos 0x20, ttl 53, id 61451, offset 0, flags
[none], proto TCP (6), length 60)
74.125.224.113.80 > XYZ.14.73.188.21613: Flags [S.], cksum 0x63ba
(correct), seq 1796974652, ack 1835043040, win 5672, options [mss
1430,sackOK,TS val 3210049129 ecr 14620543,nop,wscale 6], length 0
13:54:14.849538 IP (tos 0x20, ttl 53, id 61451, offset 0, flags
[none], proto TCP (6), length 60)
74.125.224.113.80 > XYZ.14.73.188.21613: Flags [S.], cksum 0x5f0a
(correct), seq 1796974652, ack 1835043040, win 5672, options [mss
1430,sackOK,TS val 3210050329 ecr 14620543,nop,wscale 6], length 0
13:54:15.646261 IP (tos 0x10, ttl 64, id 60781, offset 0, flags
[none], proto TCP (6), length 60)
XYZ.14.73.188.21613 > 74.125.224.113.80: Flags [S], cksum 0xb735
(correct), seq 1835043039, win 65535, options [mss 1460,nop,wscale
5,sackOK,TS val 14623543 ecr 0], length 0
13:54:15.662281 IP (tos 0x20, ttl 53, id 61451, offset 0, flags
[none], proto TCP (6), length 60)
74.125.224.113.80 > XYZ.14.73.188.21613: Flags [S.], cksum 0x5bde
(correct), seq 1796974652, ack 1835043040, win 5672, options [mss
1430,sackOK,TS val 3210051141 ecr 14620543,nop,wscale 6], length 0
13:54:17.253367 IP (tos 0x20, ttl 53, id 61451, offset 0, flags
[none], proto TCP (6), length 60)
74.125.224.113.80 > XYZ.14.73.188.21613: Flags [S.], cksum 0x55a6
(correct), seq 1796974652, ack 1835043040, win 5672, options [mss
1430,sackOK,TS val 3210052733 ecr 14620543,nop,wscale 6], length 0
^C
I tried adding an alias on em1 and I could ping things on my LAN with
ping -S <em1 alias IP>, so it seems particular to em0. I think em0 is
the onboard NIC and em1 is the PCI-Express card. Here is the relevant
pciconf output:
em0 at pci0:0:25:0: class=0x020000 card=0x849c1043 chip=0x15038086
rev=0x05 hdr=0x00
vendor = 'Intel Corporation'
class = network
subclass = ethernet
bar [10] = type Memory, range 32, base 0xfbe00000, size 131072, enabled
bar [14] = type Memory, range 32, base 0xfbe28000, size 4096, enabled
bar [18] = type I/O Port, range 32, base 0xf080, size 32, enabled
cap 01[c8] = powerspec 2 supports D0 D3 current D0
cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message
cap 13[e0] = PCI Advanced Features: FLR TP
em1 at pci0:4:0:0: class=0x020000 card=0xa01f8086 chip=0x10d38086 rev=0x00 hdr=0x00
vendor = 'Intel Corporation'
device = 'Intel 82574L Gigabit Ethernet Controller (82574L)'
class = network
subclass = ethernet
bar [10] = type Memory, range 32, base 0xfbcc0000, size 131072,
enabled
bar [14] = type Memory, range 32, base 0xfbc00000, size 524288, enabled
bar [18] = type I/O Port, range 32, base 0xe000, size 32,
enabled
bar [1c] = type Memory, range 32, base 0xfbce0000, size 16384, enabled
cap 01[c8] = powerspec 2 supports D0 D3 current D0
cap 05[d0] = MSI supports 1 message, 64 bit
cap 10[e0] = PCI-Express 1 endpoint max data 128(256) link x1(x1)
cap 11[a0] = MSI-X supports 5 messages in map 0x1c enabled
And here is the relevant dmesg output:
em0: <Intel(R) PRO/1000 Network Connection 7.1.9> port 0xf080-0xf09f
mem 0xfbe00000-0xfbe1ffff,0xfbe28000-0xfbe28fff irq 18 at device 25.0
on pci0
em0: Using an MSI interrupt
em0: [FILTER]
em0: Ethernet address: AA:BB:CC:DD:EE:FF
em1: <Intel(R) PRO/1000 Network Connection 7.1.9> port 0xe000-0xe01f
mem 0xfbcc0000-0xfbcdffff,0xfbc00000-0xfbc7ffff,0xfbce0000-0xfbce3fff
irq 18 at device 0.0 on pci4
em1: Using MSIX interrupts with 3 vectors
em1: [ITHREAD]
em1: [ITHREAD]
em1: [ITHREAD]
em1: Ethernet address: FF:EE:DD:CC:BB:AA
ifconfig em0 output:
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC>
ether AA:BB:CC:DD:EE:FF
inet XYZ.14.73.185 netmask 0xfffffff8 broadcast XYZ.14.73.191
inet XYZ.14.73.186 netmask 0xffffffff broadcast XYZ.14.73.186
inet XYZ.14.73.187 netmask 0xffffffff broadcast XYZ.14.73.187
inet XYZ.14.73.188 netmask 0xffffffff broadcast XYZ.14.73.188
inet XYZ.14.73.189 netmask 0xffffffff broadcast XYZ.14.73.189
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active
arp -na output (LAN IPs masked out):
? (XYZ.14.73.186) at AA:BB:CC:DD:EE:FF on em0 permanent [ethernet]
? (XYZ.14.73.187) at AA:BB:CC:DD:EE:FF on em0 permanent [ethernet]
? (XYZ.14.73.185) at AA:BB:CC:DD:EE:FF on em0 permanent [ethernet]
? (XYZ.14.73.190) at 00:22:2d:6c:e0:3e on em0 expires in 113 seconds [ethernet]
? (XYZ.14.73.188) at AA:BB:CC:DD:EE:FF on em0 permanent [ethernet]
? (XYZ.14.73.189) at AA:BB:CC:DD:EE:FF on em0 permanent [ethernet]
netstat -nr output:
Internet:
Destination Gateway Flags Refs Use Netif Expire
default XYZ.14.73.190 UGS 34 27763 em0
10.0.0.0/24 link#2 U 2 96232 em1 =>
10.0.0.0/8 link#2 U 1 28 em1
10.0.0.1 link#2 UHS 0 837 lo0
10.0.0.100 link#2 UHS 0 0 lo0
10.10.10.1 link#2 UHS 0 0 lo0 =>
10.10.10.1/32 link#2 U 0 0 em1
10.10.10.2 link#2 UHS 0 0 lo0 =>
10.10.10.2/32 link#2 U 0 0 em1
127.0.0.1 link#5 UH 0 486 lo0
XYZ.14.73.184/29 link#1 U 0 0 em0
XYZ.14.73.185 link#1 UHS 0 137 lo0
XYZ.14.73.186 link#1 UHS 0 0 lo0 =>
XYZ.14.73.186/32 link#1 U 0 0 em0
XYZ.14.73.187 link#1 UHS 0 0 lo0 =>
XYZ.14.73.187/32 link#1 U 0 0 em0
XYZ.14.73.188 link#1 UHS 0 0 lo0 =>
XYZ.14.73.188/32 link#1 U 0 0 em0
XYZ.14.73.189 link#1 UHS 0 0 lo0 =>
XYZ.14.73.189/32 link#1 U 0 0 em0
I can try building an 8-STABLE kernel to see if it works ok, but
ideally I'd like to remain on 8.2-RELEASE. Please let me know what
other information is needed to help debug this.
Thanks,
Josh
More information about the freebsd-net
mailing list