CARP and NIC-teaming on ESXi

[Sebastiaan van Erk]

Hi all,

I have a problem using CARP for IP failover on a FreeBSD 7.1-amd64 
virtual machine on ESXi 3.5. The problem can be reproduced using a 
single FreeBSD virtual mmachine.

When I put the FreeBSD VM on a virtual switch on ESXi with one physical 
network card associated to it, CARP works fine, the carp interface 
becomes MASTER.

However, as soon as I associate 2 physical network cards with the 
virtual switch so that I have NIC failover, the FreeBSD machine 
continuously becomes the BACKUP even though there are no other machines 
with the same vhid. In the logs I get this message repeated ad infinitum:

carp0: MASTER -> BACKUP (more frequent advertisement received)
arp_rtrequest: bad gateway 192.168.1.1 (!AF_LINK)
carp0: MASTER -> BACKUP (more frequent advertisement received)
arp_rtrequest: bad gateway 192.168.1.1 (!AF_LINK)

When I ping the shared IP addres 192.168.1.1 it replies VERY 
infrequently (a few times ever 50000 pings).

When I do a tcpdump to see the carp advertisements then when it is 
working (only one network card), I see something like this (with carp 
advertisements approximately every second):

10:08:11.241213 00:00:5e:00:01:4a > 01:00:5e:00:00:12, ethertype IPv4 
(0x0800), length 70: (tos 0x10, ttl 255, id 28744, offset 0, flags [DF], 
proto VRRP (112), length 56) 192.168.1.2 > 224.0.0.18: VRRPv2, 
Advertisement, vrid 74, prio 100, authtype none, intvl 1s, length 36, 
addrs(7): 
156.231.51.105,143.57.8.21,37.96.2.205,99.70.81.166,146.170.184.206,103.60.18.123,240.32.224.52
10:08:12.651052 00:00:5e:00:01:4a > 01:00:5e:00:00:12, ethertype IPv4 
(0x0800), length 70: (tos 0x10, ttl 255, id 28757, offset 0, flags [DF], 
proto VRRP (112), length 56) 192.168.1.2 > 224.0.0.18: VRRPv2, 
Advertisement, vrid 74, prio 100, authtype none, intvl 1s, length 36, 
addrs(7): 
156.231.51.105,143.57.8.22,34.65.134.219,132.160.185.229,103.202.156.249,174.25.227.190,231.95.30.57
10:08:14.061257 00:00:5e:00:01:4a > 01:00:5e:00:00:12, ethertype IPv4 
(0x0800), length 70: (tos 0x10, ttl 255, id 28769, offset 0, flags [DF], 
proto VRRP (112), length 56) 192.168.1.2 > 224.0.0.18: VRRPv2, 
Advertisement, vrid 74, prio 100, authtype none, intvl 1s, length 36, 
addrs(7): 
156.231.51.105,143.57.8.23,91.0.105.24,240.239.51.204,21.9.216.6,232.26.58.127,73.8.235.226

However, when I do the NIC teaming on ESXi, I constantly see 2 packets 
arrive at the same time and I get the 'more frequent advertisement 
received' message in the logs:

10:10:47.527982 00:00:5e:00:01:4a > 01:00:5e:00:00:12, ethertype IPv4 
(0x0800), length 70: (tos 0x10, ttl 255, id 30136, offset 0, flags [DF], 
proto VRRP (112), length 56) 192.168.1.2 > 224.0.0.18: VRRPv2, 
Advertisement, vrid 74, prio 100, authtype none, intvl 1s, length 36, 
addrs(7): 
156.231.51.105,143.57.8.129,248.175.196.213,178.47.150.234,200.203.153.156,219.129.15.78,19.136.6.207
10:10:47.529163 00:00:5e:00:01:4a > 01:00:5e:00:00:12, ethertype IPv4 
(0x0800), length 70: (tos 0x10, ttl 255, id 30136, offset 0, flags [DF], 
proto VRRP (112), length 56) 192.168.1.2 > 224.0.0.18: VRRPv2, 
Advertisement, vrid 74, prio 100, authtype none, intvl 1s, length 36, 
addrs(7): 
156.231.51.105,143.57.8.129,248.175.196.213,178.47.150.234,200.203.153.156,219.129.15.78,19.136.6.207

Does anybody know what could be going on, and whether it's possible to 
get this working?

Thanks in advance,
Sebastiaan

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3328 bytes
Desc: S/MIME Cryptographic Signature
Url : http://lists.freebsd.org/pipermail/freebsd-cluster/attachments/20090404/97a9793a/smime.bin