natd and ipfw external hangs

[Alex Teslik]

Hello,

    I recently upgraded to 7.0-STABLE and have setup an ipfw+natd combo on
my dual homed host. I have two interfaces:

em0 - external interface to the net 24.205.x.x
sk0 - internal interface 192.168.x.x

When users connect on the 192.168.x.x internal network everything works
great. Packets get out to the net and back to the originating machine with
no delays. So, natd seems to be doing the right thing.

The server (24.205.x.x) can directly connect to the internet for all
services - no problems there.

The problem is external users. When they hit the webserver at 24.205.x.x the
text portion of the pages load quickly. A few images load, and then the rest
of the page hangs for quite some time. When I check the connection on my
side with netstat -a I see a lot of these:

tcp4       0      0  server.http             41.221.19.24.62422
FIN_WAIT_2
tcp4       0      0  server.http             41.221.19.24.62401
FIN_WAIT_2
tcp4       0      0  server.http             203.215.120.236.1686
FIN_WAIT_2


So it seems the connection is just hanging for some reason. I opened my
firewall up completely, taking natd out of the equation and the external
problem was solved. So, I'm suspecting a bad config in my firewall rules, or
a bad config in my natd.

So I created an open firewall that also uses natd to see if I could get
things working. Here are the rules (complete with comments from the fbsd
handbook):

#!/bin/sh
IPFW="ipfw -q add"
ipfw -q -f flush

# No restrictions on Inside LAN Interface for private network
$IPFW 10 allow all from any to any via sk0

# No restrictions on loopback interface
$IPFW 20 allow all from any to any via lo0

# check if packet is inbound and nat address if it is
$IPFW 30 divert natd ip from any to any in via em0

# Allow the packet through if it has previously been added to the
# the "dynamic" rules table by an allow keep-state statement.
$IPFW 50 check-state

# Interface facing Public Internet (Outbound Section)
# Interrogate session start requests originating from behind the
# firewall on the private network or from this gateway server
# destined for the public Internet.
# Basically, let everything out.
$IPFW 60 skipto 500 all from any to any out via em0 setup keep-state

# Interface facing Public Internet (Inbound Section)
# Interrogate packets originating from the public Internet
# destined for this gateway server or the private network.
# Basically, let everything in to me.
$IPFW 70 allow all from any to me in via em0 setup limit src-addr 2

# This is skipto location for outbound stateful rules
$IPFW 500 divert natd ip from any to any out via em0
$IPFW 600 allow ip from any to any

$IPFW 800 deny all from any to any


and my natd setup:

gateway_enable="YES"
firewall_enable="YES"
firewall_script="/etc/ipfw.rules"
natd_enable="YES"
natd_interface="em0"
natd_flags="-dynamic -m"


and in my kernel:

# For Network Address Translation (NAT)
options         IPFIREWALL
options         IPFIREWALL_VERBOSE
options         IPFIREWALL_VERBOSE_LIMIT=5
options         IPFIREWALL_DEFAULT_TO_ACCEPT
options         IPDIVERT


How can I successfully eliminate the external hangs without loosing natd for
the internal users? Any ideas greatly appreciated!

Thanks,
Alex