Download speed and TCPIP window sizing

[Norberto Meijome]

On Fri, 01 Jun 2007 02:21:13 +0800
Pang <freebsd at laws.ms> wrote:

> Thanks for reply.
> 
> Nicolas Gieczewski wrote:
> > On Thu, 31 May 2007 22:06:39 +0800, Pang <freebsd at laws.ms> wrote:
> >   
> >>   Recently, I have leased a rack in Asianetcom and some bandwidth.
> >> However, the speed for my curcuit is substantially slower than my
> >> provider's company site.
> >> [...]
> >> *This server is using FreeBSD 6.2 patch 4 with Apache in the port tree.
> >> The kernel is custom built but I haven't made any modification in sysctl
> >>
> >>   I have asked many techs in Asianetcom and they blamed on TCPIP window
> >> sizing.
> >>     
> >
> > I used to have the same problem with my FreeBSD dedicated servers in the U.S. when I was living in South America with a latency of ~200 ms. Given identical hardware and connectivity, I could max out my download speed when downloading from a server running Linux, but would get no more than 100-150 KB/s when downloading from an otherwise identical FreeBSD box.
> >
> > The solution was to increase the size of the TCP send window on the FreeBSD server to about 128 KB--the default of 32 KB turned out to be way too small.
> >
> > # sysctl net.inet.tcp.sendspace=131072
> >
> > After modifying this value, you need to restart any processes (e.g. Apache) that you want to take advantage of the change.
> >   
> 
> It doesn't work. I am still getting ~10KB/s speed.
> 

hmm in some FBSD (6.0-STABLE) that we have in NL i have that set to 64K . From
AU (where I live), 320 ms of ping away, i can almost saturate my meagre DSL
link (got up to 2mpbs). 

There can be several things involved in bad download speeds - from the TCP
stack of the OS , the software you use to serve (I'm assuming Apache is
configured out of the box - it's not great, but should be better than this.),
NIC in the server, the network segment where it's located (in the DC,
upstream,etc), bandwidth limiting upstream or in apache itself,etc...

.I tested that download of yours from different parts of the world, it was
pretty bad from everywhere - I've captured the traffic from each download:

http://www.meijome.net/files/freebsd/test_hk_dl/fbsd_hk_test_from_au.cap
From AU , between 204 and 206 ms ping.

http://www.meijome.net/files/freebsd/test_hk_dl/fbsd_hk_from_NL.cap
Your file in HK downloaded from NL - ping time to HK = 329 ms

http://www.meijome.net/files/freebsd/test_hk_dl/fbsd_hk_from_txusa.cap
Your file in HK, downloaded from a linux server (RHES 3) in TX, USA. ping time
avg 227.

http://www.meijome.net/files/freebsd/test_hk_dl/fbsd_NL_test_from_au.cap
the same file, which I uploaded to a FBSD in NL (as mentioned above). Ping time
is about 320 ms to this box from home. Probably as fast as I'd expect it to be.
I know I can transfer about up of 8 Mbps between the USA and NL hosts, so
that test is not going to prove anything more.

I haven't got time right now to review all the captures, but at first sight it
seems there's more packet loss than you'd expect, duped ACKs,etc. The IO , RTT
and throughpouts graphs (i'm using wireshark) definitely show very poor
performance, though.

I dont see any relation betwen ping times and download speed.

BTW, how fast can you download this file from the server you say it's in the
same network segment ? 

Have you got polling enabled?

what hardware / nic ? 

HIH,
Beto

_________________________
{Beto|Norberto|Numard} Meijome

"It's not what you do, it's the love you put into it."
   Mother Theresa.

I speak for myself, not my employer. Contents may be hot. Slippery when wet.
Reading disclaimers makes you go blind. Writing them is worse. You have been
Warned.