Intel 10GbE Tuning under freebsd

[Jack Vogel]

First, get the latest driver if you are using the native 8.1 its way old.

Second, make sure you're in a PCI Express 2.0 slot, at least 8x, and
if you are using a consumer type system there are often slots that are
not wired with as many lanes as you might think, so verify that. Oh, and
remember, both sides of the link effect things so check both. Newer driver
versions will warn you if you don't have adequate bandwidth.

Third, make sure you have the system interrupt storm threshold increased,
hw.intr_storm_threshold, its way too low by default for 10G rates, I would
initially increase it to 10000, there will be messages about throttling if
its
not high enough, increase til you don't see them.

Fourth, make sure your mbuf pool numbers are high enough in the size that
you need. The ixgbe driver uses standard 2K clusters when using standard
MTU,
but as soon as you use jumbos it will go to 4K and then 9K as the jumbo size
increases.  increase the relevant kern.ipc.nmb...

This should get you in the ballpark, let me know if you have further
questions
or issues.

Good luck,

Jack


On Thu, May 19, 2011 at 6:52 AM, William Salt
<williamejsalt at googlemail.com>wrote:

> Hi All,
>         I have just got a couple of 10GbE intel X520-DA2 cards to test. Im
> running freebsd 8.1 on a super micro intel xeon server, and a hp core 2 duo
> workstation, both machines have 4gb of ram.
> Both are attached via sfp+ cables to a brocade turbo iron switch, which
> simply has jumbo frames enabled.
>
> Initial testing was pretty basic, i simply enabled jumbo frames, and
> increased max send/recv buffer to 16MB, which should be more than enough.
> I then ran iperf and managed to get approximately 5gbps, i was expecting
> around 8.5gbps.
> However, i realise that the ix driver probably needs to be tuned, as well
> as
> the OS.
>
> After a bit of googling, i cant find a tuning guide under freebsd for
> 10GbE,
> or many recommendations, and wondered if anyone has already successfully
> managed to tune freebsd with intel 10gbe nics, to gain a higher throughput?
> If so, has anyone got any tips, or sample configs?
>
> Thanks in advance
> Will
> _______________________________________________
> freebsd-net at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
>