Router with CARP: reproducible deadlock

[Gleb Smirnoff]

On Wed, Jan 11, 2006 at 01:53:54PM +0100, Sebastian Schwerdhoefer wrote:
S> My dream is to build a redundant router using carp. I build
S> test environments, one with FreeBSD 6.0, another one using
S> pfsense and I also tried OpenBSD 3.8.  However in every
S> environment I'm getting into a deadlock situation with the
S> following steps:
S> 
S> 1st: Set up "router1" with 2 NICs connection the networks
S> "net1" and "net2". This machine is cloned to "router2",
S> where we adjust the IP addresses on both physical
S> Interfaces. Set up carp on both machines (carp0 is
S> 192.168.0.240 and connected to net1, carp1 is 172.16.16.240
S> and connected to net2). Of course, execute 'sysctl -w
S> net.inet.carp.allow=1; sysctl -w net.inet.carp.preempt=1'
S> and connect "router2" in parallel to "router1".
S> 
S> 2nd: Start up carp on both machines, which leads to router1
S> beeing MASTER with both carp interfaces and Router2 beeing
S> BACKUP with both. Perfect for now.
S> 
S> 3rd: Unplug router1's patch cable to net1. As router1 does
S> not see any more advertisments from router2 at net1,
S> router1's carp0 stays in MASTER mode.
S> 
S> 4th: router2 doesn't see any more advertisments from router1
S> at net1 as well ('cause the cable is disconnected!). So
S> router2's carp0 becomes MASTER. Even though
S> net.inet.carp.preempt is set, router2's carp1 stays in
S> BACKUP mode.
S> 
S> 5th: Now we have the catastrophic situation:
S> 
S>     router1
S>         carp0 (net1) is in MASTER mode (cable unplugged!)
S>         carp1 (net2) is in MASTER mode
S> 
S>     router2
S>         carp0 (net1) is in MASTER mode
S>         carp1 (net2) is in BACKUP mode
S> 
S> As you can see, packets that should be routed will be lost.
S> 
S> I'm asking myself, if anybody uses carp based routers in
S> production environment? If this is the normal behaviour, you
S> should not! But I hope, that I misunderstood the carp
S> documentation and someone will enlighten me :-)
S> 
S> Any suggestions?

This means that your NIC doesn't detect loss of Ethernet link. What
NICs do you use? Please demask "net1" and "net2".

-- 
Totus tuus, Glebius.
GLEBIUS-RIPN GLEB-RIPE