rev. 1.94 of netinet/in.c broke CARP

[Robert Watson]

On Thu, 25 Jan 2007, Gleb Smirnoff wrote:

> B> >  Before this change, most of the subsystems, that allocated multicast
> B> >membership instances had freed is theirselves. I don't know about others,
> B> >but at least CARP is broken now. It attempts to free a memory, that
> B> >already has been freed.
> B> >
> B> I would suggest that the correct fix, for now, would be for carp(4) to
> B> now *not* perform its own cleanup for the IPv4 groups it joins on member
> B> interfaces.
>
> Unfortunately, this won't be a correct fix. In a scenario when the parent 
> interface stays on its place, but you are creating, attaching and destroying 
> a CARP interface, the multicast membership would not be left and memory 
> won't be freed. So, after the following sequence
>
> ifconfig fxp0 10.0.0.1/24
> ifconfig carp0 create
> ifconfig carp0 vhid 1 10.0.0.2/24
> ifconfig carp0 destroy
>
> , we would still have a multicast membership on fxp0.
>
> B> The symptom here is that carp(4) needs to join a multicast group on its 
> B> member interface. When the interface goes away, the group membership is 
> B> now destroyed, at the netinet global level, by the netinet detach path B> 
> first. B> B> However, carp(4) is keeping its own imo_membership vector of 
> the B> addresses it joined on its member interfaces (rather than using the 
> one B> which netinet assigns to it in its attach path), and later tries to 
> free B> these memberships. B> B> netinet6 does not have the same problem 
> because in6 memberships are B> reference counted. B> B> The root problem is 
> that we should be using consistent semantics for B> both the IPv4 and IPv6 
> paths, and the kernel APIs where soft-ifnets B> (such as carp(4)) and 
> routing code (such as MROUTING) need to manipulate B> multicast group 
> memberships.

Architecturally, the right fix is that CARP needs to have a handler for ifnet 
destruction that always runs before the multicast address garbage collection. 
I'm pretty preoccupied for the next few days due to an impending paper 
deadline, so can't investigate further currently, but one way or the other 
that ordering dependency needs to be expressed.  If done properly, CARP will 
always have released its multicast address before they are forceably removed. 
Having the reference count is good too, but what I describe should be 
sufficient regardless of the refcount.

Robert N M Watson
Computer Laboratory
University of Cambridge