kern/98622: [carp] carp with IPv6 broken on 6.1 (regression)

Fri Jun 23 15:50:18 UTC 2006

The following reply was made to PR kern/98622; it has been noted by GNATS.

From: Philippe Pegon <Philippe.Pegon at crc.u-strasbg.fr>
To: bug-followup at FreeBSD.org, Philippe.Pegon at crc.u-strasbg.fr
Cc:  
Subject: Re: kern/98622: [carp] carp with IPv6 broken on 6.1 (regression)
Date: Fri, 23 Jun 2006 17:41:51 +0200

 I found the culprit to be the commit dated "Fri Nov 4 20:26:14 2005 UTC"
 (cf 1.27.2.3 revision of ip_carp.c for example).

 Before this commit, everything works fine, and I can watch CARP
 advertising packets going out with IPv6 *and* IPv4 from the master.
 After this commit, only IPv4 packets are sent.

 In order to test, I built the following test-bed: 2 servers, one being
 the CARP master (s1) and the second being the slave (s2). For each
 tested version, I executed the following commands on the master:

      ifconfig carp0 create
      ifconfig carp0 vhid 219 pass carptest 130.79.201.235/24
      ifconfig carp0 inet6 2001:660:2402:1001::235/64

 and the following commands on the slave:

      ifconfig carp0 create
      ifconfig carp0 vhid 219 advskew 100 pass carptest 130.79.201.235/24
      ifconfig carp0 inet6 2001:660:2402:1001::235/64

 Before the commit:

      s1# tcpdump -eni xl0 proto carp
      tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
      listening on xl0, link-type EN10MB (Ethernet), capture size 96 bytes
      11:41:01.326923 00:00:5e:00:01:db > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: 130.79.201.192 > 224.0.0.18: VRRPv2, Advertisement, vrid 219, prio 0, authtype none, intvl 1s, length 36
      11:41:01.326984 00:00:5e:00:01:db > 33:33:00:00:00:12, ethertype IPv6 (0x86dd), length 90: 2001:660:2402:1001::236 > ff02::12: ip-proto-112 36
      11:41:02.327759 00:00:5e:00:01:db > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: 130.79.201.192 > 224.0.0.18: VRRPv2, Advertisement, vrid 219, prio 0, authtype none, intvl 1s, length 36
      11:41:02.327805 00:00:5e:00:01:db > 33:33:00:00:00:12, ethertype IPv6 (0x86dd), length 90: 2001:660:2402:1001::236 > ff02::12: ip-proto-112 36
      11:41:03.328604 00:00:5e:00:01:db > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: 130.79.201.192 > 224.0.0.18: VRRPv2, Advertisement, vrid 219, prio 0, authtype none, intvl 1s, length 36
      11:41:03.328650 00:00:5e:00:01:db > 33:33:00:00:00:12, ethertype IPv6 (0x86dd), length 90: 2001:660:2402:1001::236 > ff02::12: ip-proto-112 36
      11:41:04.329447 00:00:5e:00:01:db > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: 130.79.201.192 > 224.0.0.18: VRRPv2, Advertisement, vrid 219, prio 0, authtype none, intvl 1s, length 36
      11:41:04.329485 00:00:5e:00:01:db > 33:33:00:00:00:12, ethertype IPv6 (0x86dd), length 90: 2001:660:2402:1001::236 > ff02::12: ip-proto-112 36
      ^C
      8 packets captured
      35 packets received by filter
      0 packets dropped by kernel

 After the commit, the slave continuously attempts to become master,
 since IPv6 CARP advertisements are not sent by the master:

      s1# tcpdump -eni xl0 proto carp
      tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
      listening on xl0, link-type EN10MB (Ethernet), capture size 96 bytes
      15:05:46.976380 00:00:5e:00:01:db > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: 130.79.201.192 > 224.0.0.18: VRRPv2, Advertisement, vrid 219, prio 0, authtype none, intvl 1s, length 36
      15:05:47.977221 00:00:5e:00:01:db > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: 130.79.201.192 > 224.0.0.18: VRRPv2, Advertisement, vrid 219, prio 0, authtype none, intvl 1s, length 36
      15:05:48.978064 00:00:5e:00:01:db > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: 130.79.201.192 > 224.0.0.18: VRRPv2, Advertisement, vrid 219, prio 0, authtype none, intvl 1s, length 36
      15:05:49.978910 00:00:5e:00:01:db > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: 130.79.201.192 > 224.0.0.18: VRRPv2, Advertisement, vrid 219, prio 0, authtype none, intvl 1s, length 36
      15:05:50.979757 00:00:5e:00:01:db > 01:00:5e:00:00:12, ethertype IPv4 (0x0800), length 70: 130.79.201.192 > 224.0.0.18: VRRPv2, Advertisement, vrid 219, prio 0, authtype none, intvl 1s, length 36
      ^C
      5 packets captured
      42 packets received by filter
      0 packets dropped by kernel

 One more strange thing: when I add WITNESS and INVARIANTS options
 in the kernel (before the commit, I didn't test after it), the
 kernel panics when I add an IPv6 address to the CARP interface:

      # ifconfig carp0 inet6 2001:660:2402:1001::235/64

      panic: _mtx_lock_sleep: recursed on non-recursive mutex carp_if @/usr/src/sys/netinet/ip_carp.c:1209

      KDB: enter: panic
      [thread pid 28 tid 100024 ]
      Stopped at      kdb_enter+0x2b: nop
      db> trace
      Tracing pid 28 tid 100024 td 0xc1522600
      kdb_enter(c0894127) at kdb_enter+0x2b
      panic(c0893618,c08a1832,c08a152c,4b9,c1522678) at panic+0xbb
      _mtx_lock_sleep(c18d14d0,c1522600,0,c08a152c,4b9) at _mtx_lock_sleep+0x3a
      _mtx_lock_flags(c18d14d0,0,c08a152c,4b9,c1730aa8) at _mtx_lock_flags+0x84
      carp_macmatch6(c18d14c0,c1725700,c17306a8,c09d9460,0) at carp_macmatch6+0x27
      nd6_na_output(c1624400,c0903994,c17306a8,20,1) at nd6_na_output+0x23e
      carp_send_na(c195c200,c195c200,c195c200,c195c200,2) at carp_send_na+0x36
      carp_master_down_locked(c195c200,c18d14d0,0,c08a152c,4f4) at
      carp_master_down_locked+0x72
      carp_master_down(c195c200) at carp_master_down+0x28
      softclock(0) at softclock+0x211
      ithread_loop(c14f0400,cc9d5d38,c14f0400,c0654f50,0) at ithread_loop+0x144
      fork_exit(c0654f50,c14f0400,cc9d5d38) at fork_exit+0xa0
      fork_trampoline() at fork_trampoline+0x8
      --- trap 0x1, eip = 0, esp = 0xcc9d5d6c, ebp = 0 ---