increasing transmit speeds in WAN setting?

[Ted Mittelstaedt]

----- Original Message ----- 
From: "Moses Leslie" <marmoset at malformed.org>
To: "Ted Mittelstaedt" <tedm at toybox.placo.com>
Cc: <freebsd-questions at freebsd.org>
Sent: Friday, October 20, 2006 1:35 AM
Subject: Re: increasing transmit speeds in WAN setting?


> On Thu, 19 Oct 2006, Ted Mittelstaedt wrote:
>
> > Until you do what I told you to do and properly setup and test under
> > fxp0, I am just not going to waste my time on this anymore.  I will
> > leave you with a printout of a test run on a new mailserver I'm building
up
> > right now, in fact, using an fxp card, to prove it's a not a stack
problem.
> > You can choose to believe it or you can choose to continue wasting your
> > time chasing ghosts in the TP stack when the problem is the driver:
>
> I'm setting up test servers now, it's just taking time to get a good test
> environment up.
>
> I'll respond with actual numbers after testing, between autoneg and forced
> 100/full servers.  I admit, the forced 100/full is because of ancient
> lore, particularly with cisco switches not always playing nice with
> autonegotiation, we've just always done it that way (until gbit), and
> never had any problems.
>

Make absolutely sure to download the current catOS/IOS for your
switches, older firmware in them had problems with certain network
chipsets.  Cisco got egg on it's face - the old IOS in the 2950's would
not work with the new ethernet chipsets in the 1800/2800/3800 router
series when they came out - among other things.

> The servers in question all do 150-200Mbit in production, no problem,
> it's just that any one flow can't do more than ~300KB/s cross country.
> Given that they're over 100Mbit, what ethernet card is recommended if em
> has problems?
>

Your going to have to experiment, it's a crapshoot.  I had a hell of a time
with the bge adapter and 6.1 production, I produced a patch that helped,
finally the bge author updated the driver with a more comprehensive
fix.  It works fine now but you must get the driver from CVS, the
production 6.1 driver does not work.

I also have an em card, but I didn't do significant testing with it
after getting the bge fix.  Our largest feed is 45Mbt and so I
think it's pointless to plug a gigabit ethernet card into the network
since a 10/100 card has plenty of capability to saturate our largest
feed.  None of our switches are gigabit and it is very unlikely that
they will be upgraded in the near future.  We do not do significant
server-to-server data traffic, to be perfectly honest, I don't believe in
it.
 I come from
the school of you get 1 really big, powerful, expensive, reliable
server that has enough power to do what you need, rather than a
bunch of lame ones that are underpowered and try to cluster them.
I've never had one of these fail in production, although I've seen
a lot of clusters at customer sites that gave their admins a whole
lot of grief.

I only am dealing now with gigabit ethernet because I have to, since
it's coming standard on all the new server hardware.  And frankly I
think it sucks, since I've seen lots of problems with gigE adapters at
customer sites that were plugged into older switches.  We haven't been
bit by any of this yet - of course, we use 10/100 switches that
were top-of-the-line switches during their day - but I've personally
engineered 3 customer forklift upgrades to brand new top-of-the-line
Cisco switches due to gigabit lan negotiation and throughput problems.

Our customers have the dough to buy 80-100 ports of new Cisco
switches, (of course they think they don't - but they do) wheres like
most ISPs we don't.  And, since we don't need it anyay, what's the
point?

> FWIW, I am able to receive full speed on all of these servers.
> freebsd.org sends at 10Mbit, kernel.org at 20+.  It's only sending speed
> that I have a problem with, and only with freebsd.
>

My take on it is the gigabit ethernet chipset drivers are not completely
debugged under FreeBSD at this time.  Certainly, the Broadcom chipset
is just getting there.  The Intel chipsets usually lead the pack in support
so you probably will get more traction on complaining to the em developer
if you can demonstrate 100Mbt speeds on a fxp card, then 30Mbt speeds
on an em card, in the same machine on the same network.

FreeBSD tends to lag behind in the hardware support area.  I'm sorry about
that but you just have to accept it if your going to use FreeBSD.

Ted