Policy-based routing for packets originating from local machine ('reinject' packets back into kernel?)

m_wlist at weirdwire.ru m_wlist at weirdwire.ru
Sun Jul 22 15:24:53 UTC 2007


Hello.

I have a FreeBSD machine connected to internal network and several ISPs. I
have set up (with pf) nat and balanced (policy-based) routing for machines
from internal network, proper routing for remote connections to interfaces
that connected to corresponding ISPs.

One thing that I can't manage to get done is to make balanced routing to
work with packets originating from the router itself. Simple pf route-to
rules don't work as it seems local packets don't have any 'in' interface
-- they just emerge somewhere in kernel just before routing table and when
they are accessible to pf they are already in 'out' part where policy
routing is not possible (the packet are already going out on interface
that is determined from rather static routing table).
When there are no default route in routing table packets just have nowhere
to go and fail not even reaching any pf rules.

I suddenly came up with an idea how to try to get that done -- somehow
'reinject' packets back into kernel to get them processed by 'in' rule
with 'route-to'. The idea sounds quite simple -- make some virtual
interface with non-existing address and subnet and make default route be
somewhere in that subnet. Then make another virtual interface and
bridge/connect (with netgraph for example) it to first one the way that
any packets sent out to first one appear back in on second.

The question number one: Is there any less retarded way to do PBR for
local packets?
The question number two: How to do that properly?

At the moment I'm trying to get that working with netgraph's ngeth
interfaces. But they seem to behave in some really weird way.
Details:
# ifconfig ngeth0 10.42.42.1 netmask 255.255.255.250
# ngctl connect ngeth0: ngeth1: lower upper
# ngctl connect ngeth1: ngeth0: lower upper
('tcpdump -ni ngeth0' on other terminal for great justice)
# ping 10.42.42.2
(here after some delay I get 'host is down' messages with no output from
tcpdump).
# ping 10.42.42.5
(broadcast address, gives nothing from ping, and 'blal blah 10.42.42.1 >
10.42.42.5: ICMP echo request, blah' from tcpdump)
(here i change tcpdump from ngeth0 to ngeth1)
# ping 10.42.42.2 and # ping 10.42.42.3
give 'host is down' from ping and nothing from tcpdump
# ping 10.42.42.4
(LOL WUT!) still gives 'host is down' from ping, but tcpdump -ni ngeth1
gives 'arp who-has 10.42.42.4 tell 10.42.42.1'!

That raises two questins:
1) Wtf is going on?
2) How to make ngeth just send ip packet, avoiding that arp stuff (or is
there any other virtual interface devices available that do that)?

Seems like I don't understand something completely.



More information about the freebsd-net mailing list