FreeBSD 6.4+ PF Binat =>Degraded traffic after few hours hours.

[Aminuddin Abdullah]

I have 2 servers running FreeBSD 6.4P#1 with standard SMP and each server
has multiple IP alias bind to the bge1, Dell R200.

# ifconfig -a
bge0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500
        options=1b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING>
        ether 00:19:b9:fa:0a:9f
        media: Ethernet autoselect (none)
        status: no carrier
bge1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        options=1b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING>
        inet x.x.72.23 netmask 0xffffff00 broadcast x.x.72.255
        inet x.x.72.73 netmask 0xffffff00 broadcast x.x.72.255
        inet x.x.72.74 netmask 0xffffff00 broadcast x.x.72.255
        inet x.x.72.75 netmask 0xffffff00 broadcast x.x.72.255
        inet x.x.72.76 netmask 0xffffff00 broadcast x.x.72.255
        inet x.x.72.77 netmask 0xffffff00 broadcast x.x.72.255
        ether 00:19:b9:fa:0a:a0
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
        inet6 ::1 prefixlen 128
        inet 127.0.0.1 netmask 0xff000000
pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33208
tun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1500
        inet 10.10.10.1 --> 255.255.0.0 netmask 0xffffffff
        Opened by PID 1224

x.x.72.23 is the main IP and the rest are alias.
Tun0 is the address created by openvpn.

Following is the pf rules.

EXT_IF= "bge1"
INT_IF= "tun0"
# Configured Networks
EXT= "x.x.72.0/24"
INT= "10.10.0.0/16"
DMZ= "10.10.12.0/24"
FW= "x.x.72.23"
# DMZ Servers IP Addresses
user1="10.10.12.2"
user2="10.10.12.6"
user3="10.10.12.10"
user4="10.10.12.14"
user5="10.10.12.18"

#External IP Pool Mapping
WEB_EXT1= "x.x.72.73"
WEB_EXT2= "x.x.72.74"
WEB_EXT3= "x.x.72.75"
WEB_EXT4= "x.x.72.76"
WEB_EXT5= "x.x.72.77"


#########################################################
# NAT: Bi-directional NAT (one-to-one mapping)

binat on $EXT_IF inet from $user1 to any -> $WEB_EXT1
binat on $INT_IF inet from $user1 to any -> $WEB_EXT1
binat on $EXT_IF inet from $user2 to any -> $WEB_EXT2
binat on $INT_IF inet from $user2 to any -> $WEB_EXT2
binat on $EXT_IF inet from $user3 to any -> $WEB_EXT3
binat on $INT_IF inet from $user3 to any -> $WEB_EXT3
binat on $EXT_IF inet from $user4 to any -> $WEB_EXT4
binat on $INT_IF inet from $user4 to any -> $WEB_EXT4
binat on $EXT_IF inet from $user5 to any -> $WEB_EXT5
binat on $INT_IF inet from $user5 to any -> $WEB_EXT5

rdr pass on $EXT_IF proto {tcp, udp} from any to $WEB_EXT1 port 1024:65000
-> $user1
rdr pass on $EXT_IF proto {tcp, udp} from any to $WEB_EXT2 port 1024:65000
-> $user2
rdr pass on $EXT_IF proto {tcp, udp} from any to $WEB_EXT3 port 1024:65000
-> $user3
rdr pass on $EXT_IF proto {tcp, udp} from any to $WEB_EXT4 port 1024:65000
-> $user4
rdr pass on $EXT_IF proto {tcp, udp} from any to $WEB_EXT5 port 1024:65000
-> $user5

pass all
pass out on $EXT_IF proto {tcp,udp,icmp} from any to any keep state

-------------------------------

It's a very simple pf.rules with no block rules. Main purpose to map vpn
user to dedicated public IP.

It was working great the last few months but lately it has been giving a
terrible performance after a few hours of running the servers. SSH is not
accessible, traffic and routing is very slow.

Is the anything wrong with above configuration or 6.4 kernel with regards to
PF and OpenVPN?
The servers are not having any custom setting sysctl.conf or loader.conf or
rc.conf except the enabling openvpn, firewall and sshd.

Restarting sshd will provide remote access again or rebooting the server. Is
there any known memory leaked for pf in this configuration? Is there a
better and efficient way of doing this in PF or is it better to use ipfw?

When this happen (no ssh), all ping to the alias IPs resulted in timeout.
Only the main IP will respond.

Server RAM is 1GB and during this issue, top shows
---------------top------------
last pid:  4163;  load averages:  0.36,  0.29,  0.21
up 0+21:10:26  11:11:58
21 processes:  1 running, 20 sleeping
CPU:  2.3% user,  0.0% nice,  6.0% system,  3.9% interrupt, 87.8% idle
Mem: 15M Active, 233M Inact, 241M Wired, 76K Cache, 111M Buf, 503M Free
Swap: 1951M Total, 1951M Free
--------------------------

Anyone?

TIA.