Losing connections/performance with PF turned on

[Pat Maddox]

On 7/5/07, Greg Hennessy <Greg.Hennessy at nviz.net> wrote:
> >
> > We're doing some stress testing on our server,
>
> CPU ? Memory ?

Xeon 3060 (dual core @ 2.4 Ghz)
2 gigs of ram


> > and noticed that when
> > we turn PF on, we lose connections and have a drastic reduction in
> > performance.
> >
> > We used SIEGE for 120 seconds, 50 connections, on req/conn
> >
>
> [snip]
>
> > # --- DEFAULT POLICY
> > block log all
> >
>
> What drops are you seeing in the firewall logs for the missing connections ?

I'm not very familiar with pf at this point.  Here's a snippet of the log:

pat@~: sudo tcpdump -n -e -ttt -r /var/log/pflog | grep CLIENT
reading from file /var/log/pflog, link-type PFLOG (OpenBSD pflog file)
281. 491774 rule 2/0(match): block in on em0: CLIENT.56441 >
SERVER.80: . ack 3842266997 win 5080 <nop,nop,timestamp 995763116
242815600>
000117 rule 2/0(match): block in on em0: CLIENT.56456 > SERVER.80: P
3759758688:3759758883(195) ack 769179073 win 1460 <nop,nop,timestamp
995763116 242815600>
000007 rule 2/0(match): block in on em0: CLIENT.56442 > SERVER.80: .
ack 2278771587 win 5804 <nop,nop,timestamp 995763116 242815600>
000005 rule 2/0(match): block in on em0: CLIENT.56442 > SERVER.80: F
0:0(0) ack 628 win 5804 <nop,nop,timestamp 995763116 242815600>
000111 rule 2/0(match): block in on em0: CLIENT.56437 > SERVER.80: .
ack 21684384 win 2184 <nop,nop,timestamp 995763116 242815601>


> Are you monitoring the number of entries in the state table with pfctl -si ?
> The default is iirc 10k, a benchmarking tool can easily chew through this.
>
>
>
> Greg

I reran the benchmarks and monitored the # of entries, we hit 10k
pretty quickly.  Kept upping it until we got to 35k which is where we
stopped seeing any returns.  We still dropped some connections (99.6%
of requests came back successfully), and the throughput was 3.4 Mbp as
opposed to the 9.8 Mbps we get with the firewall off.

I'll be doing a lot more testing over the next few days, so I'll have
better info in a couple days...but if you can shed any light on this
I'd really appreciate it.

Pat