kern/121574: CARP hash dependent on order of IP addresses on interface

[Douglas K. Rand]

>Number:         121574
>Category:       kern
>Synopsis:       CARP hash dependent on order of IP addresses on interface
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Mar 10 19:10:02 UTC 2008
>Closed-Date:
>Last-Modified:
>Originator:     Douglas K. Rand
>Release:        FreeBSD 6.2-RELEASE-p7 i386
>Organization:
Meridian Environmental Technology, Inc.
>Environment:
System: FreeBSD luna-0.meridian-enviro.com 6.2-RELEASE-p7 FreeBSD 6.2-RELEASE-p7 #0: Fri Aug 24 20:46:09 CDT 2007 root@:/usr/obj/usr/src/sys/LUNA i386


>Description:
Recently I had to reboot one of my redundant firewalls and
the /etc/rc.conf file had exchanged the primary and alias
addresses of the interface. (We were switching ISPs.)

If you have a carp interface with 2 IP addresses, the IP addresses
MUST be assigned in the same order on both interfaces. Luna-0 crashed
this morning, and when I brought it back up the carp1 interface on
BOTH Luna-0 and Luna-1 were in the MASTER state.

>From Luna-0:
carp1: flags=49<UP,LOOPBACK,RUNNING> mtu 1500
        inet 12.192.93.2 netmask 0xffffffc0 
        inet 207.109.234.254 netmask 0xfffffff0 
        carp: MASTER vhid 2 advbase 1 advskew 0

>From Luna-1:
carp1: flags=49<UP,LOOPBACK,RUNNING> mtu 1500
        inet 207.109.234.254 netmask 0xfffffff0 
        inet 12.192.93.2 netmask 0xffffffc0 
        carp: MASTER vhid 2 advbase 1 advskew 50

Both were trying to act as the master carp interface, which was
causing some strange TCP errors. (Bryan noticed his automated SSH
sessions to Mars were failing.)

Once I got it straightened out, everything worked better. And of
course, once I got the addresses added to the interfaces in the right
order I realized that since the Qwest stuff is disconnected, I didn't
need the 207.109.234.254 address anymore anyway, so I could have just
deleted that, which I've now done.

Thing thing to watch for is a LOT of "carp.: incorrect hash" messages
from the kernel. I ignored the few I saw because you ALWAYS get a few
of those at boot, but I didn't realize how many were being generated.

>How-To-Repeat:
	Change the order of the IP addresses on a CARP interface from
	its peer.
>Fix:


>Release-Note:
>Audit-Trail:
>Unformatted: