IPSEC tunnel problem

Fri Apr 28 13:39:54 UTC 2006

Hello.
First of all I apologize if I freebsd-pf is not the rigth place to ask
my question. I will explain below why it is actually asked here. But if
anyone knows the better place, let me know.

On FreeBSD-6.0 I have setup IPSEC VPN tunnel as explained in the FreeBSD
Handbook -
http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/ipsec.html
I also have applied kern/91412 patch (
http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/91412 ) because it
seemed related to the issue. Unfortunately, the problem was exactly the
same before and after applying the patch.

User-visible sympthoms: a user connects to MS Remote Desktop server
through VPN tunnel and works for some time. At some random moment, RD
hangs.

tcpdump on server's side ethernet interface at that moment starts
observing ICMP host unreach packets:

(192.168.194.90 is the server while 192.168.10.176 is the client)

17:11:17.471023 IP 192.168.194.90.3389 > 192.168.10.176.4941: P
64012:65378(1366) ack 4236 win 64341 <nop,nop,timestamp 3451632
12167976>
17:11:17.496187 IP 192.168.10.176.4941 > 192.168.194.90.3389: . ack
63407 win 32409 <nop,nop,timestamp 12167976 3451632>
17:11:17.496866 IP 192.168.194.90.3389 > 192.168.10.176.4941: P
65378:66582(1204) ack 4236 win 64341 <nop,nop,timestamp 3451632
12167976>
17:11:17.497008 IP 192.168.194.90.3389 > 192.168.10.176.4941: P
66582:67376(794) ack 4236 win 64341 <nop,nop,timestamp 3451632 12167976>
17:11:17.497030 IP 192.168.194.1 > 192.168.194.90: ICMP host
192.168.10.176 unreachable, length 36
17:11:17.509615 IP 192.168.10.176.4941 > 192.168.194.90.3389: . ack
65378 win 33580 <nop,nop,timestamp 12167976 3451632>
17:11:17.512078 IP 192.168.10.176.4941 > 192.168.194.90.3389: P
4236:4253(17) ack 65378 win 33580 <nop,nop,timestamp 12167976 3451632>
17:11:17.516507 IP 192.168.194.90.3389 > 192.168.10.176.4941: P
67376:68526(1150) ack 4253 win 64324 <nop,nop,timestamp 3451633
12167976>
17:11:17.516529 IP 192.168.194.1 > 192.168.194.90: ICMP host
192.168.10.176 unreachable, length 36
17:11:17.516586 IP 192.168.194.90.3389 > 192.168.10.176.4941: P
68526:69455(929) ack 4253 win 64324 <nop,nop,timestamp 3451633 12167976>
17:11:17.516607 IP 192.168.194.1 > 192.168.194.90: ICMP host
192.168.10.176 unreachable, length 36
17:11:17.516750 IP 192.168.194.90.3389 > 192.168.10.176.4941: P
69455:70642(1187) ack 4253 win 64324 <nop,nop,timestamp 3451633
12167976>
17:11:17.516772 IP 192.168.194.1 > 192.168.194.90: ICMP host
192.168.10.176 unreachable, length 36
17:11:17.619311 IP 192.168.10.176.4941 > 192.168.194.90.3389: P
4253:4319(66) ack 66582 win 32376 <nop,nop,timestamp 12167977 3451632>
17:11:17.773334 IP 192.168.10.176.4941 > 192.168.194.90.3389: P
4319:4350(31) ack 66582 win 32376 <nop,nop,timestamp 12167979 3451632>
17:11:17.773514 IP 192.168.194.90.3389 > 192.168.10.176.4941: . ack 4350
win 64227 <nop,nop,timestamp 3451635 12167979>
17:11:17.891308 IP 192.168.10.176.4941 > 192.168.194.90.3389: P
4350:4423(73) ack 66582 win 32376 <nop,nop,timestamp 12167980 3451632>
17:11:17.997662 IP 192.168.10.176.4941 > 192.168.194.90.3389: P
4423:4475(52) ack 66582 win 32376 <nop,nop,timestamp 12167981 3451632>
17:11:17.997841 IP 192.168.194.90.3389 > 192.168.10.176.4941: . ack 4475
win 65535 <nop,nop,timestamp 3451637 12167981>
17:11:18.106066 IP 192.168.10.176.4941 > 192.168.194.90.3389: P
4475:4541(66) ack 66582 win 32376 <nop,nop,timestamp 12167982 3451632>
17:11:18.157117 IP 192.168.194.90.3389 > 192.168.10.176.4941: P
66582:67970(1388) ack 4541 win 65469 <nop,nop,timestamp 3451640
12167982>
17:11:18.157140 IP 192.168.194.1 > 192.168.194.90: ICMP host
192.168.10.176 unreachable, length 36

So, why freebsd-pf? Because I noticed in pfctl -s info output that
"state-mismatch" counter which normally is still, starts rapidly
incrementing when such a "hangups" occur. At the same time, pf should
not return ICMP messages because of

    set block-policy drop

and 

    block drop log all

as the first rule. I do not have any "block return" rules so I have no
idea who returns ICMP, why it does so and what pf counts as
state-mismatch.

The problem is 100% reproduceable and I can gather ani additional
statistics/output if it is needed.

Again, if I should ask in another place, let me know.

Regards,
Dmitry Andrianov