misc/163208: PF state key linking mismatch

[Matthew Lager]

>Number:         163208
>Category:       misc
>Synopsis:       PF state key linking mismatch
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Dec 12 16:40:07 UTC 2011
>Closed-Date:
>Last-Modified:
>Originator:     Matthew Lager
>Release:        FreeBSD 9.0-RC3
>Organization:
>Environment:
FreeBSD g03.rpsol.net 9.0-RC3 FreeBSD 9.0-RC3 #3: Fri Dec  9 15:53:39 MST 2011     mlager at g03.rpsol.net:/usr/obj/usr/src/sys/G03  i386
>Description:
With a raw IP-IP GIF tunnel set up between an 8.2-RELEASE system and an 9.0-RC3 system, the tunnel functions properly, each side can connect to eachother's network, however, the 9.0-RC3 system reports numerous PF state key linking mismatch errors, even for successful connections, that look like:

pf: state key linking mismatch! dir=OUT, if=re1, stored af=2, a0: B.B.B.B, a1: A.A.A.A, proto=4, found af=2, a0: 172.16.1.2:80, a1: 172.16.2.1:52102, proto=6.

I don't see these errors on the 8.2-RELEASE endpoint and the error seems to disrupt network performance. Here is my configuration on each endpoint, I've masked public IP addresses as A.A.A.A and B.B.B.B:

ENDPOINT 1:

/etc/rc.conf:
gif_interfaces="gif0"
gifconfig_gif0="A.A.A.A B.B.B.B"
ifconfig_gif0="inet 172.16.1.1 172.16.2.1 netmask 255.255.255.0"
static_routes="tslbell"
route_tslbell="-net 172.16.2.0/24 172.16.2.1"

/etc/pf.conf:
# MACROS
ext_if="re0"
int_if="re1"
internal_net="172.16.1.0/24"

# NORMALIZATION
scrub in all

# NETWORK ADDRESS TRANSLATION
nat on $ext_if from $internal_net to any -> ($ext_if)

# FILTERING
set skip on gif0

pass in all
pass out all

block in log all
pass quick on lo0 all
pass quick on $int_if all

# ENABLE INBOUND ICMP
pass in on $ext_if proto icmp all keep state

pass out on $ext_if proto { tcp, udp, icmp } all keep state

---------------------------

ENDPOINT 2:

/etc/rc.conf:
gifconfig_gif0="B.B.B.B A.A.A.A"
ifconfig_gif0="inet 172.16.2.1 172.16.1.1 netmask 255.255.255.0"
static_routes="belltsl"
route_belltsl="-net 172.16.1.0/24 172.16.1.1"


/etc/pf.conf:
# MACROS
ext_if="lagg0"
int_if="bge0"
internal_net="172.16.2.0/24"

# NORMALIZATION
scrub in all

# NETWORK ADDRESS TRANSLATION
nat on $ext_if from $internal_net to any -> ($ext_if)

# FILTERING
set skip on gif0

pass in all
pass out all

block in log all
pass quick on lo0 all
pass quick on $int_if all

# ENABLE INBOUND ICMP
pass in on $ext_if proto icmp all keep state

pass out on $ext_if proto { tcp, udp, icmp } all keep state
>How-To-Repeat:
Setup an IP-IP tunnel on FreeBSD 9.0-RC3, enable PF, and look for state mismatch error messages.
>Fix:
None found as of now.

>Release-Note:
>Audit-Trail:
>Unformatted: