kern/120281: Lost returning packets to PF for a rdr rule

[German M. Bravo (Kronuz)]

>Number:         120281
>Category:       kern
>Synopsis:       Lost returning packets to PF for a rdr rule
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue Feb 05 03:50:01 UTC 2008
>Closed-Date:
>Last-Modified:
>Originator:     German M. Bravo (Kronuz)
>Release:        FreeBSD 6.3-RELEASE
>Organization:
>Environment:
FreeBSD box.kronuz.com 6.3-RELEASE FreeBSD 6.3-RELEASE #11: Sat Jan 26 10:03:55 PST 2008     root at jenny6.deipi.com:/usr/obj/usr/src/sys/CUSTOM  amd64

>Description:
In a pf configuration in the firewall ("firewall"), being used to redirect and reflect connections from the outside to a box ("box1") in the inside network, and from other box ("box2") in the inside to box1, using the RDR and NAT Combination solution. The rdr rule fails to pass returning packets from box1 to the firewall whenever the connection is originated from the firewall itself.

I'm not sure if this is a bug or lacking functionality, but it would be desirable to have this working even for completeness.

>How-To-Repeat:
Set up a pf configuration involving at lest box1 and the firewall. box1 needs a running server on port tcp 25. The firewall needs the rdr and nat rules given at the bottom of this message, above the suggested topology.

Tests done:
 1. telnet to the public IP (222.66.11.33.25) from the outside world - OK
 2. telnet to the public IP (222.66.11.33.25) from box1 - OK
 3. telnet to the public IP (222.66.11.33.25) from box2 - OK
 4. telnet to the public IP (222.66.11.33.25) from the firewall itself - FAILED!

For failing test #4, the relevant tcpdump show the described behavior (em0 is the internal interface):

listening on lo0, link-type NULL (BSD loopback), capture size 96 bytes
19:35:22.751460 IP 222.66.11.33.54419 > 222.66.11.33.25: S 2677228042:2677228042(0) win 65535 <mss 1460,nop,wscale 1,nop,nop,timestamp 448892433 0,sackOK,eol>
19:35:25.871256 IP 222.66.11.33.54419 > 222.66.11.33.25: S 2677228042:2677228042(0) win 65535 <mss 1460,nop,wscale 1,nop,nop,timestamp 448895433 0,sackOK,eol>
..

listening on em0, link-type EN10MB (Ethernet), capture size 96 bytes
19:35:22.751783 IP 172.16.1.254.51628 > 172.16.1.1.25: S 2677228042:2677228042(0) win 65535 <mss 1460,nop,wscale 1,nop,nop,timestamp 448892433 0,ackOK,eol>
19:35:22.751896 IP 172.16.1.1.25 > 172.16.1.254.51628: S 1359928601:1359928601(0) ack 2677228043 win 65535 <mss 1460,nop,wscale 1,nop,nop,timestap 448857604 448892433,sackOK,eol>
19:35:22.751925 IP 172.16.1.254.51628 > 172.16.1.1.25: R 2677228043:2677228043(0) win 0
19:35:25.871270 IP 172.16.1.254.51628 > 172.16.1.1.25: S 2677228042:2677228042(0) win 65535 <mss 1460,nop,wscale 1,nop,nop,timestamp 448895433 0,ackOK,eol>
19:35:25.871475 IP 172.16.1.1.25 > 172.16.1.254.51628: S 1359928601:1359928601(0) ack 2677228043 win 65535 <mss 1460,nop,wscale 1,nop,nop,timestap 448860602 448895433,sackOK,eol>
19:35:25.871491 IP 172.16.1.254.51628 > 172.16.1.1.25: R 2677228043:2677228043(0) win 0
..

During second 22, IP 172.16.1.1.25 > 172.16.1.254.51628 shows the server in box1 is returning an ack, but then the firewall is, I think, not inverting back the rdr rule and thus telnet loses the packet and tries again (shown in second 25), as if rdr had no state for the connection.

------------------------------------
rdr and nat rules:
  rdr proto tcp from any to $ext_if port 25 tag FLOW -> 172.16.1.1
  no nat on $int_if proto tcp from $int_if to 172.16.1.1
  nat proto tcp from any to 172.16.1.1 port 25 tagged FLOW -> 172.16.1.254

------------------------------------
Suggested topology:
            Internet
               |
               | 222.66.11.33 (public IP)
          +----------+
          | firewall |
          +----------+
               | 172.16.1.254
               |
            +--+--+
 172.16.1.1 |     | 172.16.1.2
       +---_--+ +------+
       | box1 | | box2 |
       +------+ +------+

>Fix:


>Release-Note:
>Audit-Trail:
>Unformatted: