Default gateway - wrong interface. !

[Doug Ambrisko]

Gleb Smirnoff writes:
| On Sun, Feb 19, 2006 at 03:14:35PM +0000, Josef Karthauser wrote:
| J> I'm guessing that this is a bug (or feature!).
| 
| This is not a bug, nor a feature. This is a feature, that hasn't
| been implemented to the end.
| 
| Historically, the routes in kernel were static. And they are static
| now. Historically, BSD won't permit you to install same IP addresses,
| or even addresses in the same subnet, on different interfaces. Now,
| FreeBSD permits addresses in the same subnet. But route entries are
| still static, and aren't reconfigring when an interface changes its
| flags.
| 
| J> I've got a machine with a wlan interface (iwi0), with an ipv4 network
| J> address and a default gateway.  I also have an ethernet card in the same
| J> machine (em0) with the same IP address.  The idea is that I can bring
| J> the wireless down, and the wired interface up to get fast transfers when
| J> approriate, and be wireless the rest of the time.
| J> 
| J> That works fine, apart from the default gateway:
| J> 
| J>     # ifconfig iwi0 down
| J>     # ifconfig em0 up
| J>     # arp -ad
| J>     # netstat -rn
| J>     Internet:
| J>     Destination        Gateway            Flags    Refs      Use  Netif
| J>     Expire
| J>     default            87.74.4.33         UGS         0      123   iwi0
| J>     87.74.4.32/27      link#3             UC          0        0    em0
| J>     87.74.4.33         00:90:d0:02:3f:16  UHLW        2        1    em0
| J>     87.74.4.34         00:d0:b7:88:c8:20  UHLW        1  1191414    em0
| J>     127.0.0.1          127.0.0.1          UH          0        2    lo0
| J> 
| J> Notice, the local subnet is off the em0, but the default route is still
| J> wired off the iwi0.
| J> 
| J>     # route delete default
| J>     # route add default 87.74.4.33
| J>     # netstat -rn
| J>     Internet:
| J>     Destination        Gateway            Flags    Refs      Use  Netif
| J>     Expire
| J>     default            87.74.4.33         UGS         0      123   iwi0
| J>     87.74.4.32/27      link#3             UC          0        0    em0
| J>     87.74.4.33         00:90:d0:02:3f:16  UHLW        2        1    em0
| J>     87.74.4.34         00:d0:b7:88:c8:20  UHLW        1  1191414    em0
| J>     127.0.0.1          127.0.0.1          UH          0        2    lo0
| J> 
| J> The default route is _still_ off iwi0; but should be off em0.
| J> 
| J> There's obviously something dumb doing on here.  Why does the default
| J> route have to be nailed to an interface?  It's clear that 87.74.4.33 is
| J> available from em0 as far as the routing table is concerned.

FWIW, we have a patch here for 4.X that deals with this:

Index: sys/net/if.c
===================================================================
RCS file: /usr/local/cvsroot/freebsd/src/sys/net/if.c,v
retrieving revision 1.85.2.25
diff -u -p -r1.85.2.25 if.c
--- sys/net/if.c	28 Nov 2003 15:09:03 -0000	1.85.2.25
+++ sys/net/if.c	14 Oct 2004 03:49:08 -0000
@@ -553,6 +553,7 @@ ifa_ifwithaddr(addr)
 #define	equal(a1, a2) \
   (bcmp((caddr_t)(a1), (caddr_t)(a2), ((struct sockaddr *)(a1))->sa_len) == 0)
 	TAILQ_FOREACH(ifp, &ifnet, if_link)
+	  if (ifp->if_flags & IFF_UP)
 	    TAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) {
 		if (ifa->ifa_addr->sa_family != addr->sa_family)
 			continue;
@@ -578,7 +579,7 @@ ifa_ifwithdstaddr(addr)
 	register struct ifaddr *ifa;
 
 	TAILQ_FOREACH(ifp, &ifnet, if_link)
-	    if (ifp->if_flags & IFF_POINTOPOINT)
+	    if (ifp->if_flags & IFF_POINTOPOINT && ifp->if_flags & IFF_UP)
 		TAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) {
 			if (ifa->ifa_addr->sa_family != addr->sa_family)
 				continue;
@@ -617,6 +618,7 @@ ifa_ifwithnet(addr)
 	 * addresses in this address family.
 	 */
 	TAILQ_FOREACH(ifp, &ifnet, if_link) {
+	     if (ifp->if_flags & IFF_UP)
 		TAILQ_FOREACH(ifa, &ifp->if_addrhead, ifa_link) {
 			register char *cp, *cp2, *cp3;
 

I had to fix a couple of these since we up/down NIC's like you say.
We also have a local patch to make some NIC to just turn off/on the 
RX & TX stuff during a down/up so that the NIC doesn't re-initialize.

I'm not sure how well it fits in with -current and that future network
changes.  If the network guys would like this added I could do it.

Doug A.