kern/59707: IPFW2 verrevpath not working right for TCP v4?

Wed Nov 26 04:00:38 PST 2003

>Number:         59707
>Category:       kern
>Synopsis:       IPFW2 verrevpath not working right for TCP v4?
>Confidential:   no
>Severity:       serious
>Priority:       low
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Wed Nov 26 04:00:35 PST 2003
>Closed-Date:
>Last-Modified:
>Originator:     Matthias Andree
>Release:        FreeBSD 5.2-BETA i386
>Organization:
>Environment:
System: FreeBSD merlin.emma.line.org 5.2-BETA FreeBSD 5.2-BETA #1: Tue Nov 25 22:59:05 CET 2003 toor at merlin.emma.line.org:/usr/src/sys/i386/compile/MA5 i386

>Description:
ipfw * verrevpath seems not to work.
ipfw -c show (some parts masked with XXXX):

00100 174 18412 allow via lo0
00200   0     0 deny ip from any to 127.0.0.0/8
00300   0     0 deny ip from 127.0.0.0/8 to any
00400 208 45868 deny log not verrevpath in
00500   0     0 deny ip from 192.168.0.0/24 to any in via tun0
00600   0     0 deny ip from any to 10.0.0.0/8 via tun0
00700   0     0 deny ip from any to 172.16.0.0/12 via tun0
00800   0     0 deny ip from any to 192.168.0.0/16 via tun0
00900   0     0 deny ip from any to 0.0.0.0/8 via tun0
01000   0     0 deny ip from any to 169.254.0.0/16 via tun0
01100   0     0 deny ip from any to 192.0.2.0/24 via tun0
01200   0     0 deny ip from any to 224.0.0.0/4 via tun0
01300   0     0 deny ip from any to 240.0.0.0/4 via tun0
01400   0     0 deny ip from 10.0.0.0/8 to any via tun0
01500   0     0 deny ip from 172.16.0.0/12 to any via tun0
01600   0     0 deny ip from 192.168.0.0/16 to any via tun0
01700   0     0 deny ip from 0.0.0.0/8 to any via tun0
01800   0     0 deny ip from 169.254.0.0/16 to any via tun0
01900   0     0 deny ip from 192.0.2.0/24 to any via tun0
02000   0     0 deny ip from 224.0.0.0/4 to any via tun0
02100   0     0 deny ip from 240.0.0.0/4 to any via tun0
02200   0     0 check-state
02300  41  2108 deny tcp from any to any established
02400   0     0 allow tcp from any to any dst-port 22 in via tun0 setup keep-state
02500   0     0 allow tcp from any to any dst-port 113 in via tun0 setup keep-state
02600   0     0 allow udp from any to me dst-port XXXXXXXXXXXX
02700   2   100 deny log tcp from any to any in via tun0 setup
02800  85 13368 allow tcp from any to any setup keep-state
02900 216 29987 allow udp from any to any dst-port 53 out via tun0 keep-state
03000  55  4180 allow udp from any to any dst-port 123 out via tun0 keep-state
03100   2   156 deny log udp from any to any in via tun0
03200  29  2890 allow udp from any to any
03300   1    92 allow icmp from me to any keep-state
03400   1    92 allow icmp from any to any
03500   0     0 deny log ip from any to any
03600   0     0 pipe 1 out xmit tun0
65535   0     0 allow ip from any to any

Now, when I try to connect from my machine to a remote one with
"ssh user at 1.2.3.4" I'm getting loads of

| kernel: ipfw: 400 Deny TCP 1.2.3.4:22 217.225.230.222:49228 in via tun0
| kernel: ipfw: 400 Deny TCP 1.2.3.4:22 217.225.230.222:49228 in via tun0

in syslog and the counter of the 00400 rule increases, and I don't get a
connection (times out).  Leaving out the 00400 rule makes my outbound
TCP connections work.  (Apparently the 00400 rule swallows the SYN|ACK
packets.)

Note I replaced the destination address by 1.2.3.4 and the gateway
address by 217.5.*.*.

217.225.230.222 is my IP and 1.2.3.4 is the remote IP. tun0 is a PPPoE
interface, with ppp(8). I have a default route via 217.5.*.* gateway on
tun0 (both the default route and the host route for this 217.5.*.*
gateway use device tun0).

"route get 1.2.3.4" prints that 1.2.3.4 is routed via some 217.5.*.*
host which is on tun0, so this looks fine:

   route to: 1.2.3.4
destination: default
       mask: default
    gateway: 217.5.*.*
  interface: tun0
      flags: <UP,GATEWAY,DONE,STATIC>
 recvpipe  sendpipe  ssthresh  rtt,msec    rttvar  hopcount      mtu     expire
       0         0         0         0         0         0      1492         0

I'd expect that the "in via tun0" matched the outbound route as returned
by "route get".

To add to the confusion, NTP (that uses UDP) is fine, the machine will
synchronize to an outside server (my ISP's DCF receiver) via the same
gateway just fine.

Sean Chittenden asked:

> Are you on a multi-homed system?  -sc

Sort of. I do have three xl(4) NICs in my system. xl0 and xl1 are
bridged via ng_bridge, IP 192.168.0.1 on xl0, no IP on xl1;
xl2 is the transport for tun0 (which is PPPoE in my case) and
doesn't have an IP either, so "multi-homed" might read "tun0 has an
address, xl0 has another and lo0 has a third one".

I don't think this makes a difference though. verrevpath is documented
as using the interface.

>How-To-Repeat:
Set up ipfw2 as above (I'm using /etc/rc.firewall's simple section,
modified) then try to ssh to the outside as shown above.

>Fix:

>Release-Note:
>Audit-Trail:
>Unformatted: