Creating a IPSec tunnel between a Windows machine and a machine running FreeBSD

[Brent Wiese]

> 
> The topology is as fallows:
> 
> Node A <-----> Internet <------> Node B <-----> Internal net B where
> node A's external address is 111.111.11.111
> node B's internal address is 12.3.4.567
> external address is 222.222.222.2
> 
> Node A is running windows 2000 and I am using PGPnet to make
> a tunnel to Node B which is running FreeBSD and I am using 
> racoon on that machine. The tunnel has to be working for all 
> traffic. I am not sure, but I think that Node B is not set up 
> as gateway/router. 
> It is very important that the connection is set up as a 
> tunnel and not as a transporter. 
> I greatly appreciate your help. Sincerely

First, don't use PGPnet, use Win2k's built in IPSEC stuff. To get to it, do
a start/run, mmc, then file, add/remove snap-in, add. Ipsec security
policies (choose local computer), close.

Add a new policy and create the 2 tunnels. The wizards are actually
confusing so I don't use them. The key is to uncheck the "mirror" rule. It
doesn't work because of how you have to specify the tunnel end points.

You'll also need to enable Routing and Remote Access (RRAS). Make sure you
set up the filters to only allow the tunnel traffic (security thing).

After you get that set up (start/run/ipsecmon is useful) racoon is pretty
easy. You'll probably need to "enable_gateway=YES" in your rc.conf. If you
do that, you'd be wise to set up ip filtering to keep out rogue traffic. I
can't remember now if that option is actually needed for ipsec tunnels. I
know it is for MPD using pptp.

Also, don't set up the gif() stuff that you see in all the FAQ's. You only
need racoon.

If these 2 boxes aren't your default routers for the internal lans, then
you'll need to add routes to the hosts. Do not set up routes on the 2
gateway boxes, the tunnels will forward the traffic.

Ping in w2k is mostly stupid. You cannot set which NIC it sources from, so
you will not be able to ping thru the tunnel from the w2k gateway. Drive
shares and such will work. You can ping from hosts thru the tunnel though,
so use that to test.

Here is a big thing that I do not believe I've seen in the FAQ's: you need
to lower the MTU of the w2k inside NIC and all the NICs of the hosts using
the tunnel. W2k does not correctly negotiate MTU, even w/ the reg entry that
is supposed to do it set. You need to add DWORD "MTU" to
hkey_local_machine\system\currentcontrolset\services\tcpip\parameters\interf
aces and find the correct interface in that list of GUIDs. Use ping from a
host keep upping the packet size until you find the one that doesn't work,
then lower it. It'll probably be somewhere around 1420 to account for the
ipsec overhead. After creating that MTU entry, you'll need to reboot.

Good luck. You can probably tell this took an extreme amount of time to
figure out. Maybe one day I'll write my own FAQ... Hahaha!

Now that is all said and done, if what you're trying to do is give that
single w2k box access to the internal lan behind the freebsd box, you'd be
better using MPD on the unix box which supports MS's PPTP protocol. That is
much easier on the w2k box to set up. It sets up as a "dial" connection like
a modem. Its also much easier to use if the w2k box has a dynamic ip (road
warrior setup).

Brent