Outgoing packets being sent via wrong interface

[Daniel Bilik]

On Sun, 22 Nov 2015 13:02:40 +0100
Daniel Bilik <ddb at neosystem.org> wrote:

> Well, even though pf may play some role in the problem, I tend to suspect
> the routing table as the main trigger. There are several facts to support
> this...

It happened again, yesterday, and I can now definitely confirm that it's
related to default route.

In this case, affected address was 192.168.2.33. This host was unable to
connect to 192.168.2.15 (jail on the router), and router itself was unable
to even ping the affected host...

PING 192.168.2.33 (192.168.2.33): 56 data bytes
ping: sendto: Operation not permitted
ping: sendto: Operation not permitted

... because again it was pushing outgoing packets wrong way, via public
interface, where it's dropped by pf...

00:00:07.091814 rule 53..16777216/0(match): block out on re0: 82.x.y.50 > 192.168.2.33: ICMP echo request, id 12037, seq 0, length 64
00:00:01.011536 rule 53..16777216/0(match): block out on re0: 82.x.y.50 > 192.168.2.33: ICMP echo request, id 12037, seq 1, length 64

I've tried to just delete default route and enter it back to routing table.
In one tmux session ping was running, in another session I've performed
this...

# route delete default ; sleep 1 ; route add default 82.x.y.29

... and voila, ping started to communicate with affected host...

ping: sendto: Operation not permitted
ping: sendto: Operation not permitted
64 bytes from 192.168.2.33: icmp_seq=12 ttl=128 time=0.535 ms
64 bytes from 192.168.2.33: icmp_seq=13 ttl=128 time=0.264 ms

Touching nothing else (pf etc.), not rebooting, just "refreshing" the
default route entry, and the problem disappeared.

--
						Dan