Re: fib selection and persistence using ipfw

On Mon, 8 Dec 2025 08:38:22 -0500
Michael Butler <imb@protected-networks.net> wrote:

> On 12/7/25 20:11, Rozhuk Ivan wrote:
> > On Sun, 7 Dec 2025 17:28:49 -0500
> > Michael Butler <imb@protected-networks.net> wrote:
> >   
> >> Having two upstream providers, I'm trying to enforce symmetric
> >> routing which, in OpenBSD's pf config can be implemented using ..
> >>
> >> # Inbound control-plane to the firewall itself (per-WAN reply-to
> >> for symmetry)
> >> pass in on $wan_a proto { tcp, udp, icmp } to ($wan_a) \
> >>       reply-to ($wan_a $gw_a) keep state
> >> pass in on $wan_b proto { tcp, udp, icmp } to ($wan_b) \
> >>       reply-to ($wan_b $gw_b) keep state
> >>
> >> I've tried all manner of ipfw packet tagging in the hope that it
> >> would yield similar results, e.g.
> >>
> >> setfib 1 ip from any to any recv tap0
> >> setfib 1 ip from any to any tagged 1
> >> count tag 1 ip from any to any recv tap0
> >>
> >>    [ .. ]
> >>
> >> check-state
> >> allow ip from .. keep-state
> >> deny log ip from any to any
> >>
> >> Is anyone else doing something like this on -current?  
> > 
> > 
> > Actually no, but:
> > ifconfig vlan1001 172.16.0.31/24 fib 1
> > ifconfig vlan1002 172.16.0.32/24 fib 2
> > 
> > Do not forget set fib to network interface like it done in examlpe.
> > In my case if same IP+mask set on more than one net if - only last
> > one will process packets to sockets.  
> 
> Interface FIBs only work when the connection stays on the same
> machine.
> 
> In my case, I want to sustain the routing state for packets
> traversing it.
> 
> ISP-A -> Border-GW -> Mail-Server
>            ^
> ISP-B-----|
> 
> Border-GW has multiple FIBs defined and sets the relevant FIB as
> packets arrive over their respective interfaces.
> 
> Destination address is the same (Mail-Server).
> 
> When a connection is established, there is an IPFW state table entry
> in the kernel on Border-GW and which contains the FIB in 
> ipfw_dyn_rule->id->fib
> 
> What isn't happening is that replies (e.g. SYN-ACK) don't go out the 
> interface on which the SYN arrived despite having that info :-(
> 
> Is this possible with IPFW? If not, will it work with PF on FreeBSD?
> I did see some historical notes about 'reply-to' and don't know if 
> they're relevant,
>

In general, reply-to works with FreeBSD's pf. On 14.3 it also works
across multiple hosts when using pfsync (usually in combination with
carp).

Michael

-- 
Michael Gmelin