TCP connection stalled

Nagy László Zsolt gandalf at
Fri Mar 3 16:14:48 UTC 2017

Hello, I have this setup:

Remote computer <---> Internet <----->  Gateway <---->  NAT-ed local
network <---> Internal computer

The gateway is a  11.0-RELEASE-p8 running natd + ipfw.

The internet connection has download speed 500Mbps and upload speed 30Mbps.

Internet is connected via a public IPv4 address as shown below (real ip
hidden with 'x' chars)

nfe0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 40:61:86:ed:e6:41
        inet netmask 0xfffffe00 broadcast
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active

The first 4 ipfw rules are the following:

add 00005 divert natd all from any to any via nfe0
add 00102 allow all from any to any out
add 00104 allow tcp from any to any  established
add 00201 allow icmp from any to any icmptypes 0,3,8,11,12,13,14

As you can see the MTU for nfe0 was setup to 1500, and ICMP type 3 is
allowed from anybody.

If I try to copy something to a remote server with this command:

scp local_file user at remote_computer:~

Then the following happens: the first few 100K data goes through
quickly, then the connection becomes stalled. I have tried different
remote computers on different ports, but the result is always the same.
I have also tried passive mode FTP instead of SCP with the same result:

If I do the same from the internal computer behind NAT, then all uploads
and downloads are fast and responsive. The connection is stalled only
when I connect from the gateway (or to the gateway). (E.g. copy from
internal computer -> remote computer is fast and reliable.)

I was told that this might be an MTU problem. MTU discovery is turned on:

net.inet.tcp.path_mtu_discovery: 1
net.inet.sctp.pmtu_raise_time: 600

I have also tested the MTU with ping -D, the following way:

root at gw:~ # ping -D -s 1500
PING ( 1500 data bytes
--- ping statistics ---
4 packets transmitted, 0 packets received, 100.0% packet loss
root at gw:~ # ping -D -s 1400
PING ( 1400 data bytes
1408 bytes from icmp_seq=0 ttl=58 time=18.752 ms
1408 bytes from icmp_seq=1 ttl=58 time=19.502 ms
1408 bytes from icmp_seq=2 ttl=58 time=17.846 ms
1408 bytes from icmp_seq=3 ttl=58 time=16.891 ms
--- ping statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 16.891/18.248/19.502/0.978 ms
root at gw:~ #

So it seems that for this particular host, MTU=1400 sould work. So I
have changed the default MTU for this interface:

ifconfig nfe0 mtu 1400

But there was no effect at all. SCP and FTP connections are both
stalled. (I can send an example tcpdump if required, but I could find
nothing special in that.)

What else can cause this? What should I look for?



More information about the freebsd-questions mailing list