Recommendation for 1000BASE-SX card?
tedm at toybox.placo.com
Sat Jun 3 15:55:37 PDT 2006
>From: Danial Thom [mailto:danial_thom at yahoo.com]
>Sent: Friday, June 02, 2006 10:59 AM
>To: Ted Mittelstaedt; Heinrich Rebehn; questions at freebsd.org
>Subject: RE: Recommendation for 1000BASE-SX card?
>--- Ted Mittelstaedt <tedm at toybox.placo.com>
>> >-----Original Message-----
>> >From: owner-freebsd-questions at freebsd.org
>> >[mailto:owner-freebsd-questions at freebsd.org]On
>> Behalf Of Danial Thom
>> >Sent: Thursday, June 01, 2006 6:38 AM
>> >To: Heinrich Rebehn; questions at freebsd.org
>> >Subject: Re: Recommendation for 1000BASE-SX
>> >--- Heinrich Rebehn <rebehn at ant.uni-bremen.de>
>> >> Danial Thom wrote:
>> >> > The intel cards that use the EM driver
>> >> the
>> >> > best performing cards in FreeBSD that
>> >> > tested. We've test cards made by the same
>> >> company
>> >> > that use the broadcom controllers and the
>> >> intel
>> >> > cards are substantially better (ie use
>> >> CPU
>> >> > passing the same amount of traffic).
>> >> >
>> >> > Be careful using on-board controllers.
>> >> Usually
>> >> > vendors, for some reason, don't wire them
>> >> the
>> >> > pci-x bus. Most supermicro boards wire the
>> >> > controllers to the 32bit/33mhz bus and the
>> >> tyan
>> >> > and supermicro opteron boards we've tested
>> >> wire
>> >> > the broadcoms to a shared 1x PCI-E, both
>> >> which
>> >> > will not only give you poor performance,
>> >> are
>> >> > not capable of running full gigabit rates.
>> >> >
>> >> > DT
>> >> >
>> >> The Intel card would be an INTEL Pro1000MF,
>> >> right? This would be quite
>> >> expensive (~ EUR 430), but good performance
>> >> stability would warrant
>> >> that.
>> >> ATM, we are using the onboard controller
>> >> (Broadcom BCM5704C wired to the
>> >> pci-x bus). I did not have opportunity to do
>> >> performance measurements,
>> >> but we do have problems with our Linkpro
>> >> 1000SX/1000TX converters, the
>> >> 3rd of which has already died.
>> >> That's why i want to give a PCI-X card with
>> >> fiber interface a try.
>> >No, that would be the 1000MT, the MF is a
>> >card I believe. They are about US$120. in the
>> >How do you know its wired to the PCI-X bus,
>> >I don't believe that the controller has a way
>> >reporting the way that the intel controller
>> >What MB do you have?
>> >Also keep in mind that the bge driver is a
>> >of crap; driver quality is a much more telling
>> >factor in these free OS's than the card in
>> >cases. The EM and FXP are the only drivers
>> >anything (mainly because neither were written
>> >mass-driver mill man Bill Paul).
>> After having fixed bugs in the bge driver I
>> must stress
>> how wrong this statement is for the bge driver.
>> Paul may or may not have been associated with
>> the bge driver,
>> whether he was or not is immaterial since the
>> bge driver is
>> basically a port of the broadcom-supplied Linux
>> the code is Broadcoms mostly, with hunks of
>> code removed (like that dealing with the PHY's)
>> when it
>> was too difficult to port. (apparently) The
>> quality of
>> the Broadcom driver isn't Bill Paul's, it's
>> No, I can assure you that the reason the
>> chips work like crap under FreeBSD is not due
>> to Bill Paul,
>> it is because the Broadcom hardware iteself is
>> pure, unadulterated,
>> stinking, bull crap. It is crappy even under
>> the supported operating
>> systems like Windows, it's craptitude reaches
>> new heights on
>> the crap pile. Broadcom missed their calling
>> as an ethernet
>> chipset designer, they should have gone into
>> making vacuum
>> cleaners, as they would certainly be the
>> suckiest ones in
>> that business.
>I'll disagree with you on the authoring issue
>(without commenting on the crappiness of the
>controller), because it is ultimately the
>responsibility of the programmer to work around
>the quirks and even the bugs in any given
>controller, and the simple fact is that BP does a
>half-assed job; certainly not the kind of job
>someone whose sole responsibility was to maintain
>a particular driver. All complex controllers are
>a b*tch to write drivers for, and the ability to
>seemlessly integrate working code into the OS to
>mask the quirks is what separates the men from
>the boys. Saying the driver stinks because the
>example code stinks is a cop-out.
But I didn't say that. I said the driver stinks
because the HARDWARE stinks.
When I can take a Windows box with a Broadcom
chip in it, that is exhibiting timeouts and slowness,
unplug it from one brand of 10BaseT hub, and plug
it into another brand of 10BaseT hub, and then
plug my laptop into the first hub port that the
Windows box was in, and have absolutely no problems,
and have the Broadcom Windows box work perfectly in
the second brand of hub, that is crappy hardware.
It is not drivers, and no amount of twaddling with
code in the driver will fix it.
>All sample code
>stinks. The sample code should be just that; an
>example of how to program the controller.
Absolutely no, not at all. It is very easy to write
a sample driver source that is full of unexplained magic
numbers, in fact the Broadcom driver that I tweaked
was broken precisely because one of the prior
FreeBSD programmers who obviously didn't understand
teh first thing about bit mathmatics, had gone in
and replaced a bunch of magic numbers with what
he thought was their equivalents, in binary formulas.
Looking at it I know exactly what he was trying to do,
he had broken the magic numbers down to binary, seen
what appeared to be some pattern similaries between
them, tried to write some bit formulas that would
generate the same magic numbers. In short he was
assuming the magic numbers actually followed some
logic for the different chips. This was a wrong
assumption, and he didn't even do the math right.
I spent 4 hours figuring out what he had been trying
to do, and an hour chopping it out when I finally
figured out what he was trying to do, then figured
out that it wasn't calculating correctly anyway.
I could have corrected his binary formulas, but then
it would just make the driver harder to maintain since
it would no longer match the Linux driver in that
The chip datasheets are supposed to tell you how to
program the controller. And in this case, the
Broadcom-supplied Linux driver that was used as
a template is the -offically supported-
driver from Broadcom for Linux, it is not a
sample driver that came in a device drivers kit.
>example code and forcing it into a driver results
>in a garbage driver.
I will agree with this, there is plenty of that that
does happen. Not in this case, but yes, it does
>Which is also why the strategy of hiring some
>"starving programmer" is not a viable
>option,since the talent of the programmer is
>directly proportiate to the quality of the
>finished driver. The margin between marginal
>programmers (guys that can get something done
>that "works") and a superior programmer is very
>large. We're not talking about welders here.
>There could be a 50% performance difference
>between 2 drivers for the same controller written
>by programmers with different talent levels. A
>couple of fewer or more efficient I/Os, a better
>way to allocate memory, how you flip the rings;
>all can make a tremendous difference,
>particularly when you get to the 100K+ iterations
>per second levels of traffic.
The "starving programmer" was an exaggeration used
to illustrate a point, I was not seriously suggesting
to go out and hire a bad programmer.
But, when you buy cheap crappy hardware it is cheap
because the manufacturer has hired less talented
programmers among other things, and you can only
expect something that "works" not that "works well"
More information about the freebsd-questions