FreeBSD 10G forwarding performance @Intel

[Alexander V. Chernikov]

On 04.07.2012 12:13, Doug Barton wrote:
> On 07/03/2012 23:29, Alexander V. Chernikov wrote:
>> On 04.07.2012 01:29, Doug Barton wrote:
>>>>> Just curious ... what's the MTU on your FreeBSD box, and the Linux box?
>>
>> In this particular setup - 1500. You're probably meaning type of mbufs
>> which are allocated by ixgbe driver?
>
> 1500 for both?
Well, AFAIR it was 1500. We've done a variety of tests half a year ago 
with similar server and Intel and Mellanox equipment. Test results vary 
from 4 to 6mpps in different setups (and mellanox seems to behave better 
on Linux). If you're particularly interested in exact Linux performance 
on exactly the same box I can try to do this possibly next week.

My point actually is the following:
It is possible to do linerate 10G (14.8mpps) forwarding with current 
market-available hardware. Linux is going that way and it is much more 
close than we do. Even dragonfly performs _much_ better than we do in 
routing.

http://shader.kaist.edu/packetshader/ (and links there) are good example 
of what is going on.

>
> And no, I'm not thinking of the mbufs directly, although that may be a
> side effect. I've seen cases on FreeBSD with em where setting the MTU to
> 9000 had unexpected (albeit pleasant) side effects on throughput vs.
Yes. Stock drivers has this problem, especially with IPv6 addresses.
We actually use our versions of em/igb/ixgbe drivers in production which 
are free from several problems in stock driver.

(Tests, however, were done using stock driver)
> system load. Since it was working better I didn't take the time to find
> out why. However since you're obviously interested in finding out the
> nitty-gritty details (and thank you for that) you might want to give it
> a look, and a few test runs.
>
> hth,
>
> Doug
>