A simple routing question - SOLVED
smithi at nimnet.asn.au
Tue Feb 14 07:56:05 UTC 2017
In freebsd-questions Digest, Vol 662, Issue 7, Message: 6
On Sat, 11 Feb 2017 20:27:37 +0100 Robert Eckardt <rol at Robert-Eckardt.de> wrote:
> Hi again,
> thanks to all, who helped me by pointing me to some interesting
> articles on this topic.
> Eventually, I found a solution for my problem by employing
> policy based routing with layer-2 ipfw rules and multiple routing
> tables (FIBs).
> What makes things a little more complicated is the fact that both
> routers are addressed via the same interface.
> This summary is for the records for those who face the same problem.
> Using ECMP routing with kernel option RADIX_MPATH and two default
> routes proved unsuitable.
> a) since the two routers do NAT, packets must be returned through
> the same router as the connection was established and not the
> one based on destination address. (This is different from the
> situation without NAT.)
> b) After one router had failed for a short time (disconnected from
> power supply) a telnet to a host in the internet failed with
> "no route to host" while ping worked.
> I did not dig deeper into it as I concluded from several
> postings that the RADIX_MPATH feature is still unstable.
> Instead, (thanks to Matthew) I decided for policy based routing.
> Getting ipfw to filter the source MAC wasn't difficult.
> Understanding the behaviour of the stateful rules and the effect
> of layer-2 or layer-3 filtering was hard.
> Finally the following configuration works (for me):
> My SOLUTION:
> 1) since we need another forward information base, increase available
> in /boot/loader.conf
> 2) (after reboot) set alternate default routing (this needs to get set
> on every boot, so I put it in /etc/rc.local)
> setfib 1 route delete default
> setfib 1 route add default 10.0.0.2
> 3) Don't forget to also add routes to provider-specific infrastucture
> (e.g. name servers) via the correspondig router ('setfib 0' opt.)
> (This can go as a static route in /etc/rc.conf or in /etc/rc.local)
> setfib 0 route add 22.214.171.124 10.0.0.2
> 4) make sure that the logic of layer-3 rules works as before
> 2000 skipto 10000 ip from any to any not layer2
> 5) add layer-2 rule to ipfw
> 3000 allow tag 101 ip from any to any MAC any 08:9a:bc:de:ff:ff in recv re1
> 6) avoid matching of the default deny rule for layer-2 by accepting
> everything in ether_demux
> 9000 allow ip from any to any layer2
> 10000 <from here existing layer-3 rule-set>
> 7) switch on filtering of layer-2 packets
> (either in /etc/rc.local or in /etc/sysctl.conf)
If in /etc/rc.local, that obviously? needs preceding with 'sysctl '
> 8) use stateful information on layer-3
> 20000 check-state
> 20100 setfib 1 ip from any to any tagged 101 keep-state
> The incoming packet, identified on layer-2 by the MAC addres of
> Router2, is handled by the alternate FIB with the default route
> pointing back to Router2.
> 9) make sure that in the end packets get logged if some rules don't
> work as expected
> 65500 deny log ip from any to any MAC any any <-- layer-2
> 65530 deny log ip from any to any <-- layer-3
> 10) when accessing Router2, which forwards to Server, via Router1 and
> the internet, you will see working dynamic rules
> ## Dynamic rules (1 144):
> 20100 STATE tcp 126.96.36.199 51620 <-> 10.0.0.2 643
> I found (besides 'man ipfw' ;-) the following links most helpful
> Comments welcome.
Very elegant. I haven't done anything at layer2 for years, not have I
used tagging, but it smells good; happy to take your word it's working.
Would you care to post that - including your original question and
diagram below - to freebsd-ipfw at freebsd.org, where I think it would be
a) well appreciated and b) scrutinised by those who know ipfw best?
More information about the freebsd-questions