numa and taskqueues

[Emeric POUPON]

Hi,

> 
>> 2/ about https://reviews.freebsd.org/D10680, I think it would be great to have
>> this commited as a first step.
>> Since it seems to be stuck, maybe I can add more people on this. Any suggestion?
> 
> Well, what's with the ~ 8% performance decrease? Do you know what's
> going on? For a "we're parallelising IPSEC operations", seeing it get
> slower with more flows is a bit concerning.
> 
> Thanks,
> 

Actually, there is a performance boost only when few flows are involved.
That's why this is not activated by default and a sysctl is here to enable the feature.

To sum up, the more different flows you process (both ciphered and unciphered), the more network queues are hit and the more CPU units are triggered from ipsec.
In this case, we indeed notice a loss, certainly due to the extra queing/reordering performed.