Strange things on GBit / 1000->100 / net.inet.tcp.inflight.*
Raphael H. Becker
rabe at p-i-n.com
Sat Sep 18 06:54:12 PDT 2004
On Sat, Sep 18, 2004 at 03:15:07AM +0200, Andre Oppermann wrote:
> > I'm not sure this is a "physical" problem.
> The high number of out-of-order packets on your 4.10 machine is very
You're talking about
Maybe the switch is buggy for just this testcase? Store&forward-Engine?
> > Workaround:
> > * Buy 1000MBit-Cards for the both "Router"-boxes
> Why that? Do you have to do TCP transfer to/from these machines?
The two machies are routers, but ...
> Yet another test you could try:
> 5.3 GE ---> FE 4.10 "Router" FE ---> 4.10 FE
This was my initial testcase, I tried to transfer data from the GE-net
via the "internal" router to another internal FE-4.x-Box --> 700kb/sec IIRC
That failed. My first suspicion was the internal router (10.101.240.1,
4.10, FE) is not able to route a higher bandwidth for any reason. I tried
something via the router. To mask some other unknown bottlenecks I tried
direkt from the router (as target, not as router) ... now here we are.
>From my point of view ther's no significant difference between "router
just routes" or "router is target". Maybe I'm wrong here.
Just tested the case via the 2nd router to another net.
wget -O - --proxy=off ftp://10.101.240.54/pub/1GB > /dev/null
Länge: 1,160,773,632 (unmaßgeblich)
2% [=> ] 28,918,448 582.63K/s ETA 31:37^C
1 145.2xx.xxx.1x6 0.243 ms 0.120 ms 0.133 ms
2 10.101.240.54 0.357 ms 0.229 ms 0.343 ms
Same switch, different irons (target, router, server), another nets, ...
Maybe interesting: wget starts with about 1-2MBytes/sec then converging
down to some 100kBytes/sec.
Another comparison, differrent targets (4.10 FE, 5.3GE), different
servers (4.x,5.3 GE), see PS for a detailed list:
1) target: external router (10.101.240.254, 4.10, FE)
$ wget -O - --proxy=off ftp://10.101.240.52/pub/1GB > /dev/null --> 9.42M/s
$ wget -O - --proxy=off ftp://10.101.240.1/pub/250M > /dev/null --> 10.22M/s
$ wget -O - --proxy=off ftp://10.101.240.54/pub/1GB > /dev/null --> 258.47K/s
$ wget -O - --proxy=off ftp://10.101.240.55/pub/1GB > /dev/null --> 384.70K/s
$ wget -O - --proxy=off ftp://10.101.240.56/pub/1GB > /dev/null --> 195.71K/s
2) target: neighbor-server (10.101.240.55, 5.3,GE)
$ wget -O - --proxy=off ftp://10.101.240.52/pub/1GB > /dev/null --> 55.55M/s
(uncached, the .52 cannot cache 1GByte file)
$ wget -O - --proxy=off ftp://10.101.240.54/pub/1GB > /dev/null --> 34.12M/s (uncached from RAID)
$ wget -O - --proxy=off ftp://10.101.240.54/pub/1GB > /dev/null --> 76.96M/s (cached)
$ wget -O - --proxy=off ftp://10.101.240.56/pub/1GB > /dev/null --> 36.61M/s (uncached from RAID)
$ wget -O - --proxy=off ftp://10.101.240.56/pub/1GB > /dev/null --> 77.40M/s (cached)
$ wget -O - --proxy=off ftp://10.101.240.254/pub/250M > /dev/null --> 10.07M/s
$ wget -O - --proxy=off ftp://10.101.240.1/pub/250M > /dev/null --> 10.37M/s
I guess the switch has some troubles with its store&forward engine when
transferring data from GE to FE and(!) the GE is a 5.3, if GE is 4.10
then it works. So the only significant difference is the OS (thats why
I discuss it here and not with the guys of netgear this time).
I have a spare "router-box", running a offline-hot-standby copy of our
external packet filter (outside our external DMZ).
I may install 5.3 on that box to have a 5.3 running on FE, even though
I don't belive that will make a difference what kind of system is the
target. Who knows.
PS: all players on GE:
IP OS IF desc
.1 4.10 fxp1,FE internal router, PE350, 128MB RAM
.254 4.10 fxp1,FE external router, PE350, 128MB RAM
.52 4.10 bge1,GE normal host, PE2650, 1024MB RAM
.54 5.3 bge0,GE dito, 2048MB RAM
.55 5.3 bge0,GE dito, 2048MB RAM
.56 5.3 bge0,GE dito, 2048MB RAM
More information about the freebsd-current