synproxy state and route-to issues

Bill Marquette bill.marquette at gmail.com
Fri Sep 9 20:16:28 PDT 2005


I've got a machine setup with two internet facing interfaces that I want to 
do policy based routing on.

FreeBSD 6 beta 4

First two octets of the IP addresses intentionally masked.

dc0 == lan (192.168.1.1/24 <http://192.168.1.1/24>)
dc1 == isp1 (192.168.186.1/24 <http://192.168.186.1/24>)
dc2 == isp2 (192.168.104.1/24 <http://192.168.104.1/24>)

default route is set to go out isp1, each isp facing NIC is setup for NAT 
for that ISPs IP range
nat on dc1 inet from 192.168.1.0/24 <http://192.168.1.0/24> to any -> (dc1) 
round-robin
 nat on dc2 inet from 192.168.1.0/24 <http://192.168.1.0/24> to any -> (dc2) 
round-robin

 I've got a pass in rule that sets the next hop to isp2 (dc2) for any TCP 
traffic coming from one machine and uses synproxy
pass in quick on dc0 route-to ( dc2 192.168.104.1 <http://192.168.104.1> ) 
proto tcp from 192.168.1.10 <http://192.168.1.10> to any flags S/SA synproxy 
state
 
all other traffic defaults to isp1 (it all works - shown, for examples sake)
pass in quick on dc0 proto tcp from 192.168.1.0/24
<http://192.168.1.0/24>to any flags S/SA synproxy state

 A telnet from a 192.168.104.1 <http://192.168.104.1> to an internet facing 
mail server (192.168.250.25 <http://192.168.250.25>) creates the following 
state entries:

dc0 tcp 192.168.250.5:25 <http://192.168.250.5:25> <-
192.168.1.10:2592<http://192.168.1.10:2592>ESTABLISHED:ESTABLISHED
[2292384068 + 65441](+4004013808) [2512296240 + 33392](+501048536)
age 00:00:22, expires in 119:59:55, 7:4 pkts, 292:536 bytes, rule 98
id: 43210bb50000e5c8 creatorid: 65f15a74

dc1 tcp 192.168.1.10:2592 <http://192.168.1.10:2592> -> 
192.168.186.134:61140 <http://192.168.186.134:61140> ->
192.168.250.5:25<http://192.168.250.5:25>ESTABLISHED:ESTABLISHED
[3013344771 + 33397] [2292384068 + 65441]
age 00:00:22, expires in 119:59:54, 2:5 pkts, 84:580 bytes, rule 44
id: 43210bb50000e5c9 creatorid: 65f15a74

dc2 tcp 192.168.1.10:2592 <http://192.168.1.10:2592> -> 
192.168.106.121:54956 <http://192.168.106.121:54956> ->
192.168.250.5:25<http://192.168.250.5:25>SYN_SENT:CLOSED
[3013344776 + 4294967293] [0 + 65441]
age 00:00:22, expires in 00:14:45, 7:0 pkts, 292:0 bytes, rule 47
id: 43210bb50000e5cc creatorid: 65f15a74


Not totally surprising that synproxy state used the default route to send 
and create the SYN - not expected, but not surprising. You'll note that it 
went out isp1 instead of where the rule sent it to. I can live with this 
semi-unexpected behaviour...however, what ends up happening (and I don't 
have the tcpdump ready now) is that the syn, syn/ack, makes it through isp1, 
and then PF appears to hand control back to the rule processing. The ack 
from 192.168.1.10 <http://192.168.1.10> ends up going out dc2 and getting 
nat'd with dc2's IP address...thus ending any chance at the connection 
working.


Summary:
syn goes out dc1 with dc1's IP
syn/ack comes in dc1
ack goes out dc2 with dc2's IP
ack from 192.168.250.25 <http://192.168.250.25> returns on dc1 with data and 
192.168.1.10 <http://192.168.1.10> actually gets it.

 Can anyone else duplicate this? I'm suspecting that synproxy happens long 
before route-to takes place.

--Bill


More information about the freebsd-pf mailing list