misc/112565: ipfw fwd does not work with some TCP packets

Thu May 10 08:50:05 UTC 2007

>Number:         112565
>Category:       misc
>Synopsis:       ipfw fwd does not work with some TCP packets
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Thu May 10 08:50:04 GMT 2007
>Closed-Date:
>Last-Modified:
>Originator:     Michail Zhilkin
>Release:        6.2-RELEASE, from CDROM
>Organization:
SibSUTI
>Environment:
FreeBSD master 6.2-RELEASE FreeBSD 6.2-RELEASE #4: Sat May  5 14:05:35 NOVST 2007     root at master:/usr/obj/usr/src/sys/MIKE.SMP  i386

>Description:
I have a router to which several networks and DMZ are connected. 
My router has access to two different provider links 
via 802.1Q vlan interfaces: vlan10 and vlan11

System has default route through vlan11 interface. (to Provider1)

Some machines in DMZ have addresses from Provider2 network and
need to send their packets via vlan10 interface. 

I do "source routing" with ipfw for these machines:

ipfw add fwd ${provider2_GW} ip from ${DMZ_provider2_net} to not ${mylocalnet},${DMZ_provider1_net},${DMZ_provider2_net} via vlan11 out

Everything works well, but I have noticed, that some packets from ${DMZ_provider2_net} still go out through vlan11 interface.

After some analysis I have determined, that these are TCP packets with 
TCP Window = 0 and only ACK flag set.

Tcpdump on vlan11 shows:

[root at master ~]# tcpdump -i vlan11 -nn src net ${DMZ_provider2_net}

13:42:33.310277 IP 82.179.28.242.25 > 80.59.135.220.60373: . ack 2904367560 win 0
13:42:43.310267 IP 82.179.28.242.25 > 88.152.216.185.1845: . ack 289763981 win 0
13:42:43.310276 IP 82.179.28.242.25 > 200.232.191.219.56013: . ack 531945664 win 0
13:42:48.310267 IP 82.179.28.243.80 > 193.251.135.118.62076: . ack 1047166571 win 0
13:42:58.310267 IP 82.179.29.122.80 > 195.161.25.104.56876: . ack 1999970998 win 0
13:43:08.310270 IP 82.179.28.242.25 > 216.36.146.239.3639: . ack 924806561 win 0
13:43:08.310279 IP 82.179.28.242.25 > 76.169.149.45.1426: . ack 4032222169 win 0
13:43:13.310270 IP 82.179.28.242.25 > 220.36.108.40.3077: . ack 3997761968 win 0

Tcpdump on vlan10 shows:

[root at master ~]# tcpdump -i vlan10 -nn src net ${DMZ_provider2_net}

14:58:09.573966 IP 82.179.28.243.80 > 81.30.180.72.40833: . 3108981642:3108983078(1436) ack 1991712566 win 17520
14:58:09.573980 IP 82.179.28.243.80 > 81.30.180.72.40833: . 1436:2872(1436) ack 1 win 17520
14:58:09.574460 IP 82.179.28.243.80 > 81.30.180.72.40833: . 2872:4308(1436) ack 1 win 17520
14:58:10.342872 IP 82.179.28.242.53 > 206.169.72.10.43676:  9584* 1/4/5 MX mail.gasu.ru. 20 (199)
14:58:10.883958 IP 82.179.28.243.80 > 87.103.172.210.1099: . 3110871326:3110872766(1440) ack 1711239553 win 17280
14:58:12.022583 IP 82.179.28.243.80 > 80.55.114.198.1522: S 3118809207:3118809207(0) ack 332492303 win 17232 <mss 1460>
14:58:12.085034 IP 82.179.28.242.1119 > 81.19.80.12.80: S 1228111698:1228111698(0) win 1024 <mss 1460>
14:58:12.507710 IP 82.179.28.242.1119 > 81.19.80.12.80: . ack 3653374947 win 1024
14:58:12.508209 IP 82.179.28.242.1119 > 81.19.80.12.80: P 0:401(401) ack 1 win 1024
14:58:13.040301 IP 82.179.28.242.25 > 125.126.145.115.3763: . ack 1449352864 win 1024
14:58:13.065282 IP 82.179.28.243.80 > 80.55.114.198.1529: S 3119064958:3119064958(0) ack 724565672 win 17232 <mss 1460>
14:58:13.101754 IP 82.179.28.243.80 > 80.55.114.198.1531: S 3119108315:3119108315(0) ack 3810947873 win 17232 <mss 1460>
14:58:13.122738 IP 82.179.28.243.80 > 80.55.114.198.1528: S 3119144833:3119144833(0) ack 849080534 win 17232 <mss 1460>
14:58:13.148717 IP 82.179.28.243.80 > 81.30.180.72.40833: . 4308:5744(1436) ack 1 win 17520
14:58:13.311097 IP 82.179.28.242.25 > 88.247.219.128.4164: . ack 397363012 win 1024
14:58:13.451487 IP 82.179.28.243.80 > 62.183.31.9.4218: P 3113891056:3113891250(194) ack 1336319250 win 17232
14:58:13.533424 IP 82.179.28.243.80 > 81.30.180.72.40833: . 5744:7180(1436) ack 1 win 17520
14:58:14.150949 IP 82.179.28.242.1119 > 81.19.80.12.80: . ack 1 win 1024
14:58:14.216400 IP 82.179.28.242.53 > 62.220.18.13.32769:  14223* 1/4/5 MX mail.gasu.ru. 20 (199)
14:58:14.383274 IP 82.179.28.243.80 > 80.55.114.198.1510: . 3113286219:3113287655(1436) ack 259964965 win 17520
14:58:14.383286 IP 82.179.28.243.80 > 80.55.114.198.1511: . 3113250884:3113252320(1436) ack 983362080 win 17520
14:58:15.311239 IP 82.179.28.242.25 > 82.139.0.126.2728: R 1533854561:1533854561(0) win 0
14:58:15.623473 IP 82.179.28.242.1220 > 84.53.139.42.80: R 1547679025:1547679025(0) win 0
14:58:15.760368 IP 82.179.28.242.1220 > 84.53.139.42.80: R 1547679026:1547679026(0) win 0
14:58:15.781852 IP 82.179.28.242.1220 > 84.53.139.42.80: R 1547679026:1547679026(0) win 0

>How-To-Repeat:

>Fix:

>Release-Note:
>Audit-Trail:
>Unformatted: