Re: Source IPv4 address selection vs BGP IX connection

> Short version:
> 
> Using FreeBSD as a BGP router has network issues caused by suboptimal
> default IPv4 source address selection when connected to Internet
> Exchanges (which are required to use IPs that aren't routable on the
> Internet).  I was hoping to find more elegant workarounds or encourage
> FreeBSD to add source IPv4 selection akin to the existing IPv6 source
> address selection (no_prefer_iface and prefer_source).
> 
> 
> Long version:
> 
> Unless I'm mistaken, today, there is no way to set the default
> IPv4 source address for connections like there is with IPv6 (using
> no_prefer_iface and prefer_source).
> 
> It appears the default source IP is chosen based on IP address of
> the outbound interface for the packet.  This presents a problem on
> FreeBSD systems acting as BGP routers that have connections to Internet
> exchanges (IX).  One of the rules of IX IP addresses is that they are
> must not be routable on the Internet.
> 
> As a simple example, a system with two Ethernet interfaces, one to the
> transit provider and one to an IX would look like this:
> 
> vtnet0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
> 	description: Uplink
> 	inet 193.148.250.141 netmask 0xffffff00 broadcast 193.148.250.255
> vtnet1: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
> 	description: IX
> 	inet 185.1.147.211 netmask 0xffffff00 broadcast 185.1.147.255
> 
> Then if /etc/resolv.conf contains 8.8.8.8 and BGP selects a route for
> 8.8.8.0/24 over the IX, you end up with:
> 
> # route  -n get 8.8.8.8
>    route to: 8.8.8.8
> destination: 8.8.8.0
>        mask: 255.255.255.0
>     gateway: 185.1.147.22
>         fib: 0
>   interface: vtnet1
>       flags: <UP,GATEWAY,DONE,PROTO1>
>  recvpipe  sendpipe  ssthresh  rtt,msec    mtu        weight    expire
>        0         0         0         0      1500         1         0
> 
> And DNS on the system doesn't work as all DNS requests go out with a
> source address of 185.1.147.211 (the IX endpoint) which isn't exported
> as an Internet route.
> 
> While I can set a static route for 8.8.8.8 for this particular case, it
> would be messy to have to set up static routes for every possible local
> connection (other DNS servers, outbound SMTP for periodic/cron mail,
> etc.).

The mistake your making, IMHO, is that an IX connected eBGP FreeBSD
router _SHOULD NOT_ be doing ANYTHING other than BGP on the IX
connected interface, and anything like DNS and outbound SMTP should be
going inward on the AS, not outward to the internet.

I must ask why your using 8.8.8.8 and not your own nameservers?
Why would you want or even allow outbound SMTP from such a
critical infustructure point go out over the unwashed internet?

One of the reasons for using the non-routable IP on IX connected
eBGP routers is to minimize the exposure footprint, and what you
seem to be doing is defeating that minimization by wanting to
expose another IP on that very box to the public internet.

> 
> I assume that there is a group of BGP enthusiasts using FreeBSD lurking
> on freebsd-net.  What have you done to solve this problem?

I only trust AS internal objects from my eBGP routers,
they have no need to speak to the unwashed internet other
than to IX peers.
 
> I'd also love to hear other tips for running BGP on FreeBSD.

Lock it down as tight as you can if your IX connected.
I dont even allow inbound BGP connection setup, all eBGP
sessions have to be initiated by my router.

ipfw -a list 20179
20179     23854      1131316 deny log tcp from any to any 179

This is at an ISP peer, not an IX, so not a private IX IP range,
but 23854 attempts to connect to my bgp.

-- 
Rod Grimes                                                 rgrimes@freebsd.org