fastforward/routing: a 3 million packet-per-second system?

[Andrey V. Elsukov]

On 24.07.2014 16:47, John Jasen wrote:
> On 07/24/2014 05:24 AM, Andrey V. Elsukov wrote:
>> On 22.07.2014 19:18, John Jasen wrote:
>>> Feedback and/or tips and tricks more than welcome.
>>>
>>> Outstanding questions:
>>>
>>> Would increasing the number of processor cores help?
>> AFAIR, increasing the number of cores will lead to worse results.
>> With patched and tuned FreeBSD we able to route (with fastforwarding)
>> about 7 Mpps IPv4 and 2.5Mpps IPv6. But the stock system is far from
>> even half of this results.
>>
> 
> Increasing the physical CPU count can (and probably will) result in
> performance degradation. However, from what I've seen, balancing IRQs
> across the cores on a single physical CPU seems to help.
> 
> I am curious as well, as to how you achieved 7 Mpps. Can you share the
> system specs, the patches and tuning?

Usually the systems have 2x8 cores (e.g. Intel Xeon E5-2660), 16-32G
RAM, 2 ixgbe(4) network cards (82599EB). We slowly trying merge our
patches to FreeBSD, but most of them are hackish and needs rethinking.
We need reduce lock contention in routing code and optimize structures
to fit cache lines. Some of patches were published here by melifaro at .
Some are already committed. Also v6-related patches included in my svn
branch user/ae/inet6.

Tuning depends from your hardware. It is good to pin network adapter's
taskq threads and interrupts. andre@ committed this functional recently
in head/.

-- 
WBR, Andrey V. Elsukov