FreeBSD bridge + filtering, BIG problem

[Daniel Hartmeier]

On Thu, Dec 02, 2004 at 12:25:57PM +0900, Pyun YongHyeon wrote:

> Are you sure you can see *states* with "pfctl -ss"?
> Both pf/ipf can't create states since it couldn't see ANY outbound
> packets in bridge environments. In jail(fw01), you can see states
> since packets go through L3 hook points.

pf can very well create states on a FreeBSD bridge, they just don't work
properly. The problem is not creating the state entry, but that pf does
not see all packets involved in the connection in both directions.

For example, if you have a bridge with two real interfaces xl0 and xl1,
and a TCP connection passing through the bridge (say, from the xl0
network to the xl1 one).

As I understand it, pf filtering only happens for packets coming in on
xl0 and coming in on xl1, but not going out on xl0 or going out on xl1.

The initial SYN packet of the connection will first pass in on xl0,
where it's being filtered by pf. If you have a last-matching rule like

  pass in on xl0 proto tcp ... keep state

the packet will successfully create a state entry and gets passed. This
state entry will also match any further packets of the connection coming
in on xl0 (i.e. packets from the active peer who opened the connection).

But packets flowing in the reverse direction (replies from the passive
peer) are not filtered by pf on xl0, pf never sees outgoing packets on
xl0, due to the missing hook. Hence, no such packets are associated with
the state entry, and the state entry never becomes fully established.
The sequence number tracking code in pf relies on seeing all packets of
a connection. For instance, it reads the passive peer's initial sequence
number (ISN) from the SYN+ACK packet, and uses acknowledgments sent by
the passive peer to advance the window of allowed segments for the
active peer. Missing all these packets from the passive peer, the state
entry never gets completed, and the connection eventually stalls. Some
features (like 'modulate state') will even prevent a successful
handshake in this case.

If you're using 'floating states' (the default) instead of
interface-bound states, you might expect that packets from the passive
peer seen incoming on xl1 should resolve the problem. But that's not the
case. Replies coming in on xl1 do NOT match the state entry created by
the SYN on xl0, because the direction (incoming) and the
source/destination addresses/port (from passive to active) do not match.

Imagine you had all hooks, i.e. pf would see each packet on each
interface. If the bridge consists of just two interfaces, you can filter
on only one of them, as each packet passing in or out on one interface
must also pass on the other, so you could just use

  pass quick on xl1
  block all
  pass in on xl0 proto tcp ... keep state

That would work fine. Packets (of either direction) on xl0 would create
or match states, packets on xl1 would pass without matching states.

If you want to filter on both interfaces, you'd use

  block all
  pass in on xl0 proto tcp ...criteria A... keep state
  pass out on xl1 proto tcp ...criteria A... keep state
  pass in on xl1 proto tcp ...criteria B... keep state
  pass out on xl0 proto tcp ...criteria B... keep state

Each connection passing through the firewall would create TWO state
entries, one on each interface. Each packet related to a connection
would first match one state when incoming on one interface, then match
the other state on the second interface.

But either way, pf needs to see packets passing through an interface in
both directions. It does not contain hardcoded assumptions like "this is
a bridge, treat incoming packets on xl1 as if they were outgoing on xl0"
or the like.

You can check what packets pf sees in what direction on which interfaces
by using a single rule like

  pass log all

and watching pflog with tcpdump.

The OP quoted fully established state entries, which I wouldn't have
expected. I don't think this is supposed to work at all on bridges yet.
It's not solvable by a configuration trick, it really needs the missing
hooks in bridge code. I didn't know about that sysctl for ipfilter,
though, that might affect things for pf.

Daniel