Interface address sourced packets go thru default gateway on another interface

[Dima Dorfman]

Brian Hawk <brian at tnetus.com> wrote:
> since it shouldn't really happen and it used not to happen.
> Everything was working fine until I don't know when and why, now I
> cannot send any packets out thru my xl1 interface by binding its
> source address to the packets.

I don't think it ever worked the way you described. The source IP
address doesn't usually affect how replies will be routed on the way
out.

You can fix this with policy routing rules. Here's an example with PF:

: pass out quick route-to ($other_if $other_gw) from ($other_if)

$other_if is the name of the interface and $other_gw is the name of
the gateway through that interface. You need to do this for every
interface other than the one used by the default gateway. The rule
says: If the packet is coming from an IP address assigned to
$other_if, then send it through $other_gw. If you use stateful
inspection, you need corresponding reply-to rules in the other
direction:

: pass in quick reply-to ($other_if $other_gw) inet proto tcp to ($other_if) port ssh keep state

This idiom is useful on systems with multiple indepenent Internet
connections. With these rules, failure of the primary connection will
not prevent full connectivity through the secondary.

--
Dima