No ifp->if_ioctl for SIOCAIFADDR and SIOCDIFADDR causes CARP issues on 7.3?

[Manish Vachharajani]

I was poking around some of the in_carp.c code and noticed some odd
behavior due to the fact that ifp->if_ioctl is not always called on
address configuration.
It seems that ifp->if_ioctl is not called for SIOCAIFADDR and
SIOCDIFADDR, only SIOCSIFADDR under FreeBSD 7.3 release.  I confirmed
this with a few debug printfs.  This results in a few bugs:


1)  Improper advertisement even when no IPs are configured

Since carp_del_addr is what decrements sc->naddrs, carp will continue
to advertise (with the correct HMAC, if there is only one IP, since
the carp_hmac_prepare is not called on address deletion) even when
there are no IP addresses on the interface.  As a result, you can end
up with a master system that will not actually serve for the given IP
when packets arrive.  To cause this,  just do

ifconfig carp0 create
ifconfig carp0 vhid 10 advskew 0
ifconfig carp0 <some ip address on a subnet on an up interface>
ifconfig -a # confirm that the carp0 interface is MASTER
ifconfig carp0 <same ip address as above> delete
tcpdump -i <physical interface with subnet above> proto carp #notice
the advertisements continue
ifconfig carp 0 destroy #now the advertisements stops and the physical
interface leaves promiscuous mode, confirm via dmesg and above tcpdump

Note further that whatever physical interface is associated with the
carp device will stay in promiscuous mode until the carp interface is
destroyed for the same reason.


2)  Improper operation with multiple IPs per CARP interface if IPs are
added in the wrong order

Carp will work with one virtual ip per carp interface but causes it to
function improperly with multiple IPs.  For example, it appears that
carp will compute the wrong HMAC if IPs are added in different orders
on different machines since the hmac is only updated for the first IP
address in the group (set with SIOCSADDR by ifconfig).  However,
carp_hmac_prepare will do the right thing, if only it were called on
SIOCAIFADDR and DIFADDR.

3) Not interoperable with multiple IPs (compared to other CARP implementations)

Any CARP implementation (such as OpenBSD's which appears to do the
right thing) will not interoperate with FreeBSD's (though I haven't
confirmed this by booting OpenBSD and trying it) if multiple IPs are
configured because FreeBSD only computes the HMAC based on the first
IP address that was added, even though carp_hmac_prepare has code to
correctly recompute the hmac when new IPs are added in an address
independent fashion.  It is just that the function is never called
since the update would normally be through ifp->if_ioctl.

According to the man page for ifnet(9), this is the correct behavior,
if_ioctl is only called for SIOCSIFADDR, not AIFADDR and DIFADDR.  How
should a driver go about listening for these events if not through
ifp->if_ioctl?

Manish