Issue with routing table entries, jails and pf filtering on loopback interfaces

Thomas Steen Rasmussen thomas at
Fri Jun 19 07:11:19 UTC 2015

Hash: SHA1
Hello list,

This will be a long post, apologies, but it is a complex issue. First I
will explain how the server is configured, then I will explain the
problem and the workaround I found.

When I add one or more IP aliases to a non-loopback interface, lagg0 in
this example, FreeBSD adds two routing table entries per IP:       link#6             UHS         lo0    link#6             U         lagg0

The first entry, the one with "lo0" as it's interface is the problem.

The server also has an extra loopback interface with rfc1918 addresses
for jails that do not need a real, public IP. A pretty common setup.

I also run pf on the server, first filtering rule is "block log all" so
I explicitly need to permit traffic on every interface (I do
"set skip on lo0" though). I permit all outgoing traffic both on
lo1 and on lagg0, so only incoming traffic needs to be permitted.

The problem is that with this setup an rfc1918 jail _cannot_ establish a
TCP connection to a service listening on an IP on the lagg0 interface.
I think this is because the "lo0" routing table entry forces the traffic
through an interface where it logically should not appear.

The symptom:
When I try to ssh from a jail with an rfc1918 IP (say on lo1
to a jail with IP on lagg0, then pf blocks the traffic,
but not as I'd expect. I'd expect it to block an incoming packet on
lagg0 from the rfc1918 ip to the jailhosts IP. Instead it blocks the
syn/ack packet, but in the wrong direction, check this tcpdump from pflog0:

06:51:38.188170 rule 0..16777216/0(match): block out on lo1: > Flags [S.], seq 573771477, ack
2565048197, win 65535, options [mss 16344,nop,wscale 6,sackOK,TS val
3503997415 ecr 53572243], length 0

This traffic should never appear in this direction on this interface. No
wonder it gets blocked.

The workaround:
I've been able to workaround this issue by deleting the "lo0" routes for
the IPs on lagg0. If I try SSH again everything looks correct again:

06:57:44.590356 rule 0..16777216/0(match): block in on lagg0: > Flags [S], seq 3762725019, win 65535,
options [mss 1460,nop,wscale 6,sackOK,TS val 53938646 ecr 0], length 0

The syn packet is blocked where I'd expect it to get blocked, and if I
permit to SSH to in pf.conf it works as expected.

I've been asking around and it seems like I'm not the only one with this
problem. People just do "set skip on lo1" and never notice it. From a
security perspective though I really don't want traffic between jails
to be unfirewalled.

Can anyone shine a light on this? Thanks a bunch in advance :)

Best regards

Thomas Steen Rasmussen

ps. For what it's worth OpenBSD adds an lo0 route for IP aliases on
real interfaces too. No idea if the same problem is present though.
Version: GnuPG v2.0.22 (MingW32)

More information about the freebsd-net mailing list