PF, bridge, states and window scaling problem

[Alupului Costin]

On Nov 13, 2007 4:20 AM, Girish Venkatachalam
<girishvenkatachalam at gmail.com> wrote:
> On 22:08:03 Nov 12, Alupului Costin wrote:
> > I seem to have quite a problem with PF. I have set up a bridge to
> > shape my upstream traffic. I use ALTQ with hfsc discipline; but that's
> > not really important. My problem comes with the filter rules. I have
> > to use keep state because of the speed benefits (really I don't have a
> > choice),
>
> One should always keep state.
>
> > but PF has a problem when the clients passing traffic through
> > the bridge use TCP window scaling. Here is an example of four filter
> > rules that I thought should work to pass the traffic from one client
> > through the bridge and create a state:
> >
> > pass in quick on vlan0 from any to anIP/32
> > pass out quick on vlan0 from anIP/32 to any keep state queue ul_client
> > pass in quick on vlan1 from anIP/32 to any
> > pass out quick on vlan1 from any to anIP/32 keep state queue dl_client
> >
> > The above rules generate state-mismatches.
>
> Didn't get you. What sort of mismatch?

When that client tries logging in to Yahoo Messenger I can see an
increase in the number of state-mismatch reported by pfctl -si. There
are states established, but after a while the packets simply do not
match the states created. Also they will not create new states and nor
will they match a catch-all rule which follows.

>
> > I thought that would be
> > because pf doesn't see the SYN packet, although it does (one of the
> > out rules) and should create the state then... I tried writing all the
> > rules with keep state (even the inbound ones) but then nothing would
> > work at all. My intention was to create if-bound states, but I
> > switched back to floating states in the hope that pf would associate
> > the state created by an outbound rule with the traffic returning on
> > another interface of the bridge; still didn't work.
> >
>
> Have you tried adding "flags S/SAFR" to the filter rules?
>
> Try it and let me know.

I have tried using "flags S/SA" with the filter rules. The result was
that states were created, but not matched by the rest of the packets
in the stream. Packets would just match a catch-all rule that follows
the above mentioned rules. Still it was better because the connection
wouldn't just stall, but after all that was not statefull inspection
anymore...

>
> > I have read the man page for if_bridge and set the following sysctl variables:
> >
> > net.link.bridge.pfil_onlyip: 1
> > net.link.bridge.pfil_bridge: 0
> > net.link.bridge.pfil_member: 1
> >
> > I have also read some posts on the web that said that pf simply
> > doesn't have all the hooks necesary to do the filtering inbound and
> > outbound, but reading the pfil man page I seem to disaggree with that.
> >
>
> What do you mean? ?

I mean that according to the pfil and if_bridge man pages pf should
have all the hooks necesary to filter the packet inbound and outbound
on both (/all) members of the bridge, provided that I have set up the
sysctl variables as mentioned above.

>
> > Has anyone encountered the same problem? And, more important: if i
> > give up the bridge setup and switch to routing, would that have any
> > effect? I.E: will I then be able to use keep state with the inbound
> > rules?
>
> Try it. Routing changes the topology a good deal. But I doubt if that is
> the issue here. No harm in testing though.

I have tried the same setup (without the queues) on a router and I
used keep state on all the rules (even the inbound ones). Works
perfectly. So I guess the problem really is the bridge. In that case I
would kindly suggest that the pf.conf manual page should mention that
statefull firewall has an unpredictable behaviour on bridges. I.E. you
can not create states on inbound rules at all although filtering
works. Another problem is that states created by outbound traffic
don't seem to take into account the window scaling when the client
uses that.
I was a big fan of the bridge setup simply because it is transparent
and I would choose the bridge over the router setup anytime, provided
that it would work properly (i mean statefull firewall).

>
> >
> > Any help at all would be hugely appreciated as I am trying for about a
> > week to sort out this problem and can't seem to get any closer. The
> > only solution was to kindly ask my clients using TCP window scaling
> > (Vista mostly) to turn off this feature... Now I am seriously
> > considering bumping my bridge to a router but I am not sure that the
> > problem will be solved then.
>
> Try adding the flags switch as mentioned above. That way the states get
> established only from a TCP Syn packet.
>
> You should also try flushing the old states using pfctl(8).

I always flushed the old states over and over again. The flags did not
help me. As I mentioned earlier they did establish the connection on
the SYN packet, but the rest of the packets in the flow did not match
that connection.

>
> >
> > Oh, here is the setup of the bridge from rc.conf, although there
> > shouldn't be any problems there (the bridge works fine without pf, or
> > with pf stateless):
>
> Stateful filtering is always recommended. Performance is not the only
> reason why you should use it.
>
> It also adds to security. Have you tried disabling normalization/scrub?

Have tried without normalization, without fragment reassembly, with
no-df... Pretty much all the combinations...

I will answer here to Erik Osterholm also:

Performance really is an issue here when I give up statefull
inspection. The firewall contains roughly 2000 filter rules and the
traffic passing through is 20kpps at peak hours. So it is a huge
difference between statefull and stateless filtering. If I drop the
stafefull filtering the machine simply cannot handle all the traffic,
or in the best case scenario it develops quite some latency.

>
> Best,
> Girish
> _______________________________________________
> freebsd-questions at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-questions
> To unsubscribe, send any mail to "freebsd-questions-unsubscribe at freebsd.org"
>