Losing connections/performance with PF turned on

[Pat Maddox]

We're doing some stress testing on our server, and noticed that when
we turn PF on, we lose connections and have a drastic reduction in
performance.

We used SIEGE for 120 seconds, 50 connections, on req/conn

Firewall On:

Num Users: 50
Availability: 97.23 %
Transaction rate: 58.02 trans/sec
Concurrency: 3.80
Successful: 6994
Failed: 198
Longest Req: 9.06s

Firewall Off:

Num Users: 50
Availability: 100 %
Transaction rate: 94.62 trans/sec
Concurrency: 1.76
Successful: 11342
Failed: 0
Longest Req: 0.51s



You'll notice that with the firewall off, we don't lose any
connections.  We also have a 60% increase in tx/sec, and the longest
request takes 1/20 of the time.

I'm including my complete pf.conf file in the hopes that something
will jump out at someone.

Thanks,
Pat




# ------- pf.conf skeleton for server
#
# --------------- MACRO Section  -----------------

EXT_IF="em0"

PING = "echoreq"

# --- allowed incoming services initiated by clients

TCP_IN = "{ http, 8080, nrpe }"
#UDP_IN = "{ }"

SSH_IN = "{ xxx.xxx.xxx.xxx }"

# --- allowed services initiated by server

TCP_OUT = "{ ssh, smtp, domain, ntp, 3690, 2222, http, ftp, 29125 }"
UDP_OUT = "{ domain, ntp }"

ORIGINS = "{ xxx.xxx.xxx.xxx }"

# ------------------ TABLE Section --------------

# ------------------ OPTIONS Section
set loginterface $EXT_IF
set block-policy return

# --------- TRAFFIC NORMALIZATION ----------------
scrub in all
antispoof for $EXT_IF

# ---------- TRANSLATION Section (NAT/RDR)

# ---------- FILTER section

# --- DEFAULT POLICY
block log all

# --- LOOPBACK
pass quick on lo0 all

# --- BRUTE FORCE TABLE
table <bruteforce> persist
block quick from <bruteforce>

# ======================= INCOMING ================
# ----------- EXTERNAL INTERFACE

# --- TCP
pass in quick on $EXT_IF inet proto tcp from any to $EXT_IF port
$TCP_IN flags S/SA keep state

# Allow postgres connection from db server
pass in quick on $EXT_IF inet proto tcp from xxx.xxx.xxx.xxx to
$EXT_IF port 5432 flags S/SA keep state

# --- SSH with brute force blocking
pass in quick on $EXT_IF inet proto { tcp, udp } from any to $SSH_IN
port ssh flags S/SA keep state (max-src-conn 25, max-src-conn-rate
5/3, overload <bruteforce> flush global)

# --- UDP
#pass in quick on $EXT_IF inet proto udp from any to $EXT_IF port
$UDP_IN keep state

# --- ICMP
pass in quick on $EXT_IF inet proto icmp from any to $EXT_IF icmp-type
$PING keep state

# ======================= OUTGOING ================
# ----------- EXTERNAL INTERFACE

# --- TCP
pass out quick on $EXT_IF inet proto tcp from $EXT_IF to any port
$TCP_OUT flags S/SA  keep state

# Allow postgres connection to db server
pass out quick on $EXT_IF inet proto tcp from $EXT_IF to
xxx.xxx.xxx.xxx port 5432 flags S/SA keep state

# Allow any connection to uploaders
pass out quick on $EXT_IF inet proto tcp from $EXT_IF to $ORIGINS
flags S/SA keep state

# --- UDP
pass out quick on $EXT_IF inet proto udp from $EXT_IF to any port
$UDP_OUT keep state

# --- ICMP
pass out quick on $EXT_IF inet proto icmp from $EXT_IF to any
icmp-type $PING keep state

# ----------------- end of pf.conf