Problems with NAT on gif interface for VPN (long post)

Wed Oct 27 11:13:16 PDT 2004

All,
    I'm having a problem getting nat to work on a gif interface. My
goal here is to have a FreeBSD host (which is the gateway for a home
network) connect to a VPN using a "client vpn" setup and masquerade
(nat) the network behind the FreeBSD host using a single IP provided
by the corporate VPN server.
   At this point I have racoon/ipfw/natd working to the point that I
can pass traffic over the VPN from the FreeBSD host itself just fine.
However, if I send traffic from a client on the LAN, it goes all the
way across the VPN, I see the destination host send a response and I
see the incoming ESP packet on the FreeBSD hosts wan interface, but I
never see the packet sent out the LAN and I'm having trouble tracking
where it is being dropped. If I perform a packet capture on gif0, I
see neither the outgoing or incoming traffic, which seems incorrect to
me since I do know the traffic is going out and is coming back when I
initiate it from the FreeBSD host itself.

I have started two natd processes bound to two different ports for
divert use on each interface, they are shown below.

Anyways, here are the details:

Host A (FreeBSD)
LAN: 192.168.1.1/24
WAN: a.b.c.d (dynamic IP from cable provider)

Host B (VPN server)
LAN: 10.1.8.12/24 (larger /8 behind this)
WAN: w.x.y.z

VPN client IP is 10.1.13.100

---

setkey -DP:
w.x.y.z[any] a.b.c.d[any] ip4
        in ipsec
        esp/tunnel/w.x.y.z-a.b.c.d/require
        spid=67 seq=3 pid=94282
        refcnt=1
10.0.0.0/8[any] 10.1.13.100[any] any
        in ipsec
        esp/tunnel/w.x.y.z-a.b.c.d/require
        spid=69 seq=2 pid=94282
        refcnt=1
a.b.c.d[any] w.x.y.z[any] ip4
        out ipsec
        esp/tunnel/a.b.c.d-w.x.y.z/require
        spid=66 seq=1 pid=94282
        refcnt=1
10.1.13.100[any] 10.0.0.0/8[any] any
        out ipsec
        esp/tunnel/a.b.c.d-w.x.y.z/require
        spid=68 seq=0 pid=94282
        refcnt=1

---

Relevant routing entries
Destination        Gateway            Flags    Refs      Use  Netif Expire
10                 10.1.8.12          UGSc        1     9098   gif0
10.1.8.12          10.1.13.100        UH          1        0   gif0

---

ifconfig:

xl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        inet6 fe80::260:8ff:fea5:d095%xl0 prefixlen 64 scopeid 0x1
        inet a.b.c.d netmask 0xfffffe00 broadcast 255.255.255.255
        ether 00:60:08:a5:d0:95
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
vr0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
        inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255
        inet6 fe80::250:8dff:fe5d:c6d%vr0 prefixlen 64 scopeid 0x2
        inet 192.168.1.2 netmask 0xffffffff broadcast 192.168.1.2
        ether 00:50:8d:5d:0c:6d
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
gif0: flags=28151<UP,POINTOPOINT,RUNNING,PROMISC,MULTICAST> mtu 1280
        tunnel inet a.b.c.d --> w.x.y.z
        inet6 fe80::260:8ff:fea5:d095%gif0 prefixlen 64 scopeid 0x8
        inet 10.1.13.100 --> 10.1.8.12 netmask 0xffffffff

---

natd processes (latter is setup for debugging):

/sbin/natd -dynamic -use_sockets -unregistered_only -n xl0
/sbin/natd -v -l -n gif0 -log_denied -p 8669

---

ipfw show

(Yes, this set isn't that clean - it's been hacked up while trying to
figure out this issue - please forgive any redundancies or
ineficiencies you see unless they may impact this problem)

00100 1458262 852524563 divert 8668 ip from any to any via xl0
00200    3436    370488 divert 8669 ip from any to any via gif0
00300       0         0 check-state
00400  890895 403006370 allow ip from any to any keep-state out xmit xl0
00500       0         0 allow ip from any to any keep-state via gif0
00600       4       432 allow esp from any to any
00700       0         0 allow udp from any to me 500 in
00800    2770   5371008 allow ip from any to any via lo0
00900 1364895 918328465 allow ip from any to any keep-state via vr0
01000    6699   1015786 allow tcp from any to any 22 keep-state in
01100   44273  13145900 allow tcp from any to any 25 keep-state in
01200    3361   1924613 allow tcp from any to any 80 keep-state in
01300     392     26364 allow icmp from any to any
01400   55996  18161747 deny log ip from any to any
65535     104     11992 deny ip from any to any

--

So, at this point when I send traffic from a machine on the LAN, I can
confirm the following:

NAT seems to work based on the debug output of natd:
Out [ICMP] [ICMP] 192.168.1.101 -> 10.1.2.115 8(0) aliased to
           [ICMP] 10.1.13.100 -> 10.1.2.115 8(0)

I see the icmp packet reach the other side of the VPN via tcpdump, and
I see an icmp echo response sent back. If I sniff my xl0 interface
while doing this I see the following:

11:03:34.093263 a.b.c.d > w.x.y.z: ESP(spi=0xa9c55011,seq=0x5)
11:03:34.128605 w.x.y.z > a.b.c.d.167: ESP(spi=0x0dcf2aab,seq=0x5)

These packets correspond w/ the ICMP request/response I see on the
other side of the VPN so I assume that 2nd ESP packet is the ping
response.

My problem is that I never see any icmp on gif0 (neither the ping
request or response) and I never see the icmp response on vr0.
Obviously the machine on the LAN never recieves it either.

Questions:
1) Why would I not be seeing all traffic over the VPN on gif0? It
doesn't seem likely that it's going via another interface because the
source of traffic is correct and NAT seems to be working outbound, so
I think I should see it.

2) Are there problems with my configuration which anyone can identify
as preventing only the response traffic back into this network?

If you need additional info I'm happy to provide it - but I thought
this was a good (and definitely long enough) starting point. If there
is another list better suited to answering this questions let me know.

Thanks in advance,
Aaron