Carp seems completely broken on 8.2-RC2 and 8.2-PRERELEASE

Paul Schenkeveld freebsd at psconsult.nl
Sun Jan 16 12:41:31 UTC 2011


Hi,

Trying to upgrade two Soekris firewalls to 8-STABLE or 8.2-PRERELEASE
it appears that carp doesn't work at all.  I've set up carp like I've
done on many firewall pairs before and they all work correctly.  With
google, nor in the mailing lists, I could find anything about changes
in the way carp get configured but if I missed something I'd be happy
to hear that it's my fault.

Here's the setup:

		  net5501
		   test3
		10.4.0.4/24
		     |
	-------------+-------------
	 |                       |
      net4801                 net4801
       test1                   test2
 sis4: 10.4.0.2/24       sis4: 10.4.0.3/24
 carp4:10.4.0.1/24       carp4:10.4.0.1/24
   |   |   |   |           |   |   |   |
   |   |   |   |           |   |   |   |
 sis[0-3] connected to other networks, see
 explanation below.

When I ping from test3 to 10.4.0.1, I see the following traffic using
tcpdump:

    test3 # tcpdump -e -n -i vr3 not vrrp
    tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
    listening on vr3, link-type EN10MB (Ethernet), capture size 96 bytes
    12:09:35.121831 00:00:24:c9:30:ff > ff:ff:ff:ff:ff:ff,
		    ethertype ARP (0x0806), length 60:
		    Request who-has 10.4.0.1 tell 10.4.0.4, length 46
    12:09:35.122144 00:00:24:c3:49:91 > 00:00:24:c9:30:ff,
		    ethertype ARP (0x0806), length 60:
		    Reply 10.4.0.1 is-at 00:00:5e:00:01:68, length 46
    12:09:35.122173 00:00:24:c9:30:ff > 00:00:5e:00:01:68,
		    ethertype IPv4 (0x0800), length 98:
		    10.4.0.4 > 10.4.0.1: ICMP echo request,
		    id 40482, seq 0, length 64

    test1 # tcpdump -e -n -i sis4 not vrrp
    tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
    listening on sis4, link-type EN10MB (Ethernet), capture size 96 bytes
    12:09:34.977570 00:00:24:c9:30:ff > ff:ff:ff:ff:ff:ff,
		    ethertype ARP (0x0806), length 60:
		    Request who-has 10.4.0.1 tell 10.4.0.4, length 46
    12:09:34.977705 00:00:24:c3:49:91 > 00:00:24:c9:30:ff,
		    ethertype ARP (0x0806), length 42:
		    Reply 10.4.0.1 is-at 00:00:5e:00:01:68, length 28

    test2 # dump -e -n -i sis4 not vrrp
    tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
    listening on sis4, link-type EN10MB (Ethernet), capture size 96 bytes
    12:09:35.090050 00:00:24:c9:30:ff > ff:ff:ff:ff:ff:ff,
		    ethertype ARP (0x0806), length 60:
		    Request who-has 10.4.0.1 tell 10.4.0.4, length 46

There is an ARP request which is replied to by the carp master (test).
the ping to the carp address does not even appear on the sis4 interface
of test1.

This is the kernel config for test1 and test2:

    include         GENERIC
    device          carp
    makeoptions     MODULES_OVERRIDE=""

The relevant rc.conf bits:

on test1
    hostname="test1"
    cloned_interfaces="carp1 carp2 carp3 carp4"
    ifconfig_sis0="xxx.xxx.xxx.41/26"
    ifconfig_sis1="10.1.0.2/24"
    ifconfig_sis2="10.2.0.2/24"
    ifconfig_sis3="10.3.0.2/24"
    ifconfig_sis4="10.4.0.2/24"
    ifconfig_carp1="10.1.0.1/24 vhid 101 pass abcd1234 advskew   0"
    ifconfig_carp2="10.2.0.1/24 vhid 102 pass abcd1234 advskew   0"
    ifconfig_carp3="10.3.0.1/24 vhid 103 pass abcd1234 advskew   0"
    ifconfig_carp4="10.4.0.1/24 vhid 104 pass abcd1234 advskew   0"

on test2
    hostname="test2"
    cloned_interfaces="carp1 carp2 carp3 carp4"
    ifconfig_sis0="xxx.xxx.xxx.42/26"
    ifconfig_sis1="10.1.0.3/24"
    ifconfig_sis2="10.2.0.3/24"
    ifconfig_sis3="10.3.0.3/24"
    ifconfig_sis4="10.4.0.3/24"
    ifconfig_carp1="10.1.0.1/24 vhid 101 pass abcd1234 advskew 100"
    ifconfig_carp2="10.2.0.1/24 vhid 102 pass abcd1234 advskew 100"
    ifconfig_carp3="10.3.0.1/24 vhid 103 pass abcd1234 advskew 100"
    ifconfig_carp4="10.4.0.1/24 vhid 104 pass abcd1234 advskew 100"

In /etc/sysctl.conf:
    net.inet.carp.preempt=1

Ifconfig output:

test1 # ifconfig sis4
sis4: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=83808<VLAN_MTU,WOL_UCAST,WOL_MCAST,WOL_MAGIC,LINKSTATE>
        ether 00:00:24:c3:49:91
        inet 10.4.0.2 netmask 0xffffff00 broadcast 10.4.0.255
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
test1 # ifconfig carp4
carp4: flags=49<UP,LOOPBACK,RUNNING> metric 0 mtu 1500
        inet 10.4.0.1 netmask 0xffffff00
        carp: MASTER vhid 104 advbase 1 advskew 0

test2 # ifconfig sis4
sis4: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=83808<VLAN_MTU,WOL_UCAST,WOL_MCAST,WOL_MAGIC,LINKSTATE>
        ether 00:00:24:c3:49:7d
        inet 10.4.0.3 netmask 0xffffff00 broadcast 10.4.0.255
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
test2 # ifconfig carp4
carp4: flags=49<UP,LOOPBACK,RUNNING> metric 0 mtu 1500
        inet 10.4.0.1 netmask 0xffffff00
        carp: BACKUP vhid 104 advbase 1 advskew 100

There are no packet filters in place, sis1, sis2 and sis3 are wired
through cross-cables from test1 to test2, so no traffic there except for
carp.  The sis4 interfaces and vr3 of test3 are on a dumb switch with no
other stuff connected.

Setting net.inet.carp.log=7 does not result in any console/dmesg/messages
output.

I see carp traffic on sis4 which appears normal except that I don't
understand the addrs(7): part but that used to be there on 8.0/8.1
firewalls too:

    12:26:52.387140 00:00:5e:00:01:68 > 01:00:5e:00:00:12,
		    ethertype IPv4 (0x0800), length 70:
		    (tos 0x10, ttl 255, id 61070, offset 0, flags [DF],
		    proto VRRP (112), length 56)
	            10.4.0.2 > 224.0.0.18: VRRPv2, Advertisement,
		    vrid 104, prio 0, authtype none, intvl 1s, length 36,
		    addrs(7): 198.145.25.33,1.75.182.226,80.169.106.108,
		    170.107.157.42,147.165.174.125,42.254.15.27,182.184.82.166

    12:26:53.387903 00:00:5e:00:01:68 > 01:00:5e:00:00:12,
		    ethertype IPv4 (0x0800), length 70:
		    (tos 0x10, ttl 255, id 61479, offset 0, flags [DF],
		    proto VRRP (112), length 56)
	            10.4.0.2 > 224.0.0.18: VRRPv2, Advertisement,
		    vrid 104, prio 0, authtype none, intvl 1s, length 36,
		    addrs(7): 101.233.35.135,163.243.214.16,230.125.241.59,
		    123.57.190.52,104.246.131.251,255.69.201.65,61.158.20.122

Regards,

Paul Schenkeveld


More information about the freebsd-stable mailing list