10gigE link with FreeBSD hosts ?

[Barney Cordoba]

--- On Fri, 12/11/09, Jack Vogel <jfvogel at gmail.com> wrote:

> From: Jack Vogel <jfvogel at gmail.com>
> Subject: Re: 10gigE link with FreeBSD hosts ?
> To: "George Sanders" <gosand1982 at yahoo.com>
> Cc: freebsd-net at freebsd.org
> Date: Friday, December 11, 2009, 9:51 PM
> Can't get 10 gigabit thru a PCI-X
> slot, it doesnt have the bandwidth, you
> need PCI Express, in
> fact you will find that lots of factors will come into play
> and might keep
> you from achieving
> optimal thruput.
> 
> For a two port adapter you will need an 8x PCIE slot,
> preferably Gen 2, and
> if all three systems
> don't have that its gonna slow everyone down.
> 
> Further, once you involve packet forwarding it gets even
> more demanding.
> 
> Trying to do it 'on the cheap' and you might get 3 or 4 Gb,
> maybe even worse
> depending.
> 
> Takes quality to get real performance, sorry :)
> 
> Jack
> 
> 
> On Fri, Dec 11, 2009 at 2:30 PM, George Sanders <gosand1982 at yahoo.com>wrote:
> 
> >
> >
> > I am seeing Sun Microsystems branded 10gigE adaptors
> (pci-x) on ebay
> > nowadays for $500.
> >
> > So ... let's say I took some plain-jane, modern-ish PC
> (intel core, or c2d,
> > pci-x, etc.) and set up this topology:
> >
> >
> > a <-- cable --> b  <-- cable --> c
> >
> > and on each of (a) and (c) put one 10gigE adaptor, and
> on (b) put two
> > 10gigE adaptors, and set up bridging on (b)
> >
> > Would it be reasonable to then expect to transfer data
> from (a) to (c) at
> > a, roughly, 10 gigabit/s rate ?
> >
> > If not, what is the limiting factor ?
> >
> > The idea is that I would use a commodity PC + multiple
> pci-X slots as a
> > "poor mans" 10gigE switch ... I can't afford an actual
> 10gigE switch, but I
> > don't need that many ports anyway.
> >
> > Any comments or field reports would be appreciated.
> >
> >

Supermicro makes some cheap cards that are under $500. Part# is
AOC-STG-i2 and they work with FreeBSD. You should be able to find
them for around $450, so if you just need a 2 port switch its fairly
inexpensive.

The problem is that the FreeBSD ixgbe driver can't do close to 10Gb/s, 
so its not a very good switch. 

Barney