Optimizing a high-latency connection

Brian Candler B.Candler at pobox.com
Tue Aug 29 11:16:23 UTC 2006


On Fri, Aug 25, 2006 at 03:46:22PM +0200, Morgan wrote:
> I'm trying som file transfers across the globe. The RTT is almost 400ms and
> the transfer rate is painfully slow. There are 24 router hops on the path
> and I assume most of the problem is there

The number of routers isn't an issue, as long as they forward packets
without dropping them on the floor.

> but I wonder if there are any
> sysctl variables I can trim on my side to help this problem slightly? I seem
> to understand that there need to be an ACK received for at least every other
> packet

No, the two sides negotiate a 'window' which is the amount of outstanding
unacknowledged traffic in transit. tcpdump will show you the negotiated
window size. As long as the window is greater than the (bandwidth * RTT)
product, you should be able to fill the pipe.

e.g. a 2Mbps pipe (256KBps) with 400ms RTT has a delay-bandwidth product of
0.4*256 = 102.4KB

You need to define "painfully slow", but my guess is that your problems are
due to packet loss. Each lost packet causes the window size to be reduced by
half, and it will only gradually creep back up. This is part of TCP's
congestion control mechanism, since TCP equates packet loss with congestion,
indicating that the transmission speed must be reduced.

You can check your packet loss like this:

    # ping -s1472 -c100 x.x.x.x

where x.x.x.x is the remote endpoint. (This sends 1500-byte sized packets:
20 bytes IP header + 8 bytes ICMP header + 1472 bytes data padding).

TCP performs extremely poorly at packet loss of around 2% and above.

If you see a large level of packet loss, by changing x.x.x.x to point to
intermediate routers shown by traceroute, you may be able to identify where
the packet loss is taking place.

An ethernet full-duplex/half-duplex mismatch can cause packet loss in the 2%
range, which is not enough for the network to be visibly "broken", but
enough for it to perform very badly. Worst offenders here are old Cisco
Catalyst 29xx switches, which don't perform auto-negotation properly.

Regards,

Brian.


More information about the freebsd-net mailing list