BAD state/State failure with large number of requests

[Rolf Grossmann]

Hi,

I'm trying to use pf in a load-balancing setup, but it's giving me touble.
So I have set up a simple test scenario that looks like this (not really
balancing anything ;)):

      Client -- Balancer        -- Server
 10.25.0.100    em0: 10.25.0.41
                em1: 10.1.1.1      10.1.1.2

On the server I'm running apache (on port 8080) with a static page.
On the Client, I'm running Apache's ab benchmark to generate a lot of requests.

Now, if have a route to 10.1.1.2 and I'm running
   ab -n 10000 -c 100 http://10.1.1.2:8080/index.html.en
(i.e. not using the nat/rdr rule, see below) everything works fine.

However, if I'm running
   ab -n 10000 -c 100 http://10.25.0.41:8080/index.html.en
it hangs around between 1000 and 3000 requests and on the balancer I
get lots of messages like those:

Sep 28 23:56:56 balancer kernel: pf: BAD state: TCP 10.1.1.2:8080 10.25.0.41:8080 10.25.0.100:52209 [lo=2341692840 high=2341759447 win=33304 modulator=0 wscale=1] [lo=2919421554 high=2919488162 win=33304 modulator=0 wscale=1] 9:9 S seq=2345137961 ack=2919421554 len=0 ackskew=0 pkts=6:5 dir=in,fwd
Sep 28 23:56:56 balancer kernel: pf: State failure on: 1       | 5

(Values for lo, high, seq  and ack (obviously) and sometimes pkts are varying.)

I've been searching the 'net and looking at similar reports, but so far
I haven't found any suggestions. I've of course increased the states limit
to handle the number of requests (see ruleset below), however I can't
figure out if it's a bug or a limit or what else I'm running into here.

Anyone, help, please?

Thanks, Rolf.

Here's the pf.conf:

# tables (for load balancing)
table <server> persist { 10.1.1.2 }
table <balanced> persist { 10.25.0.41 }

# options
set optimization conservative
set block-policy drop
set limit states 200000
set debug misc

# load balancing
rdr inet proto tcp to <balanced> port 8080 -> <server> port 8080 round-robin

# filter rules

# we're not trying to be a firewall, so just allow all traffic
pass all