pf eates syn packet?

[Frank Behrens]

I've been using pf for some weeks (after some years with ipfw) without problems, but see a 
weird effect now - pf(4) seems to eat a tcp syn packet.  ;-)

This setup works without problems:
- FreeBSD 6.2-STABLE-200705211513 i386
- acts as router, some internal interfaces, one WAN interface (tun0, DSL with pppoe)
- NAT for WAN interface

The problem appears when
- additional WAN interface added (tun2, also DSL with pppoe)
- policy based routing added (to test for http port only)

# pfctl -s nat
nat-anchor "iface2" all
nat on tun0 inet from <intern> to any -> tun0-address

# pfctl -a iface2 -s nat
nat inet from !tun2-address to any port = http -> tun2-address

pass out quick on tun0 route-to (tun2 tun2-peer) inet from tun2-address to any keep state
pass out quick on tun2 route-to (tun0 tun0-peer) inet from tun0-address to any keep state


When I try to connect from internal (NATed) host to an external address I see a delay, 
because the 1st SYN is resent (on internal interface):
13:55:30.256823 IP (tos 0x0, ttl 128, id 35958, offset 0, flags [DF], proto: TCP (6), length: 52) 192.168.50.02.2923 > 193.99.144.85.80: S, cksum 0x3f22 (correct), 1489020152:1489020152(0) win 65535 <mss 1460,nop,wscale 1,nop,nop,sackOK>
13:55:33.266554 IP (tos 0x0, ttl 128, id 35967, offset 0, flags [DF], proto: TCP (6), length: 52) 192.168.50.02.2923 > 193.99.144.85.80: S, cksum 0x3f22 (correct), 1489020152:1489020152(0) win 65535 <mss 1460,nop,wscale 1,nop,nop,sackOK>
13:55:33.325734 IP (tos 0x0, ttl 249, id 7928, offset 0, flags [DF], proto: TCP (6), length: 52) 193.99.144.85.80 > 192.168.50.02.2923: S, cksum 0xc2b3 (correct), 3368657865:3368657865(0) ack 1489020153 win 4320 <mss 1440,nop,wscale 0,sackOK,eol>
13:55:33.325857 IP (tos 0x0, ttl 128, id 35968, offset 0, flags [DF], proto: TCP (6), length: 40) 192.168.50.02.2923 > 193.99.144.85.80: ., cksum 0x6b49 (correct), ack 1 win 43008
13:55:33.326854 IP (tos 0x0, ttl 128, id 35969, offset 0, flags [DF], proto: TCP (6), length: 137) 192.168.50.02.2923 > 193.99.144.85.80: P 1:98(97) ack 1 win 43008

then the traffic is normal, without any anomaly.

On outgoing interface tun2 I see:
13:55:33.266603 IP (tos 0x0, ttl 127, id 35967, offset 0, flags [DF], proto: TCP (6), length: 52) 84.182.234.162.58104 > 193.99.144.85.80: S, cksum 0xfd03 (correct), 1489020152:148902015       2(0) win 65535 <mss 1460,nop,wscale 1,nop,nop,sackOK>
13:55:33.325695 IP (tos 0x0, ttl 250, id 7928, offset 0, flags [DF], proto: TCP (6), length: 52) 193.99.144.85.80 > 84.182.234.162.58104: S, cksum 0x8095 (correct), 3368657865:3368657865       (0) ack 1489020153 win 4320 <mss 1440,nop,wscale 0,sackOK,eol>
13:55:33.325880 IP (tos 0x0, ttl 127, id 35968, offset 0, flags [DF], proto: TCP (6), length: 40) 84.182.234.162.58104 > 193.99.144.85.80: ., cksum 0x292b (correct), ack 1 win 43008
13:55:33.326872 IP (tos 0x0, ttl 127, id 35969, offset 0, flags [DF], proto: TCP (6), length: 137) 84.182.234.162.58104 > 193.99.144.85.80: P 1:98(97) ack 1 win 43008


So the 1st SYN packet seems to disappear, that creates an additional delay on every 
connection.

Now I my questions come:
1. Does anybody know the reason, is there an error in my setup?
2. Do you believe it is pf(4) related or is it an network stack problem?
3. What can I do to solve the problem? Some more debug output? I can test patches or 
include some debug code in my kernel.

Regards,
   Frank
-- 
Frank Behrens, Osterwieck, Germany
PGP-key 0x5B7C47ED on public servers available.