IPSEC in GENERIC [was: Re: netmap in GENERIC, by default, on HEAD]

[Bjoern A. Zeeb]

On 06 Nov 2014, at 01:10 , George Neville-Neil <gnn at neville-neil.com> wrote:

> On 5 Nov 2014, at 9:20, Alexander V. Chernikov wrote:
> 
>> On 05.11.2014 19:39, George Neville-Neil wrote:
>>> Howdy,
>>> 
>>> Last night (Pacific Time) I committed a change so that GENERIC, on HEAD has the netmap
>>> device enabled.  This is to increase the breadth of our testing of that feature prior
>>> to the release of FreeBSD 11.
>>> 
>>> In two weeks I will enable IPSec by default, again in preparation for 11.
>> Please don't.
>> 
>> While I love to be able to use IPSEC features on unmodified GENERIC kernel, simply enabling
>> IPSEC is not the best thing to do in terms of performance.
>> 
>> Current IPSEC locking model is pretty complicated and is not scalable enough.
>> It looks like it requires quite a lot of man-hours/testing to be reworked to achieve good performance and I'm not sure
>> if making it enabled by default will help that.
>> 
>> Current IPv4/IPv6 stack code with some locking modifications is able to forward 8-10MPPS on something like 2xE2660.
>> I'm in process of merging these modification in "proper" way to our HEAD, progress can be seen in projects/routing.
>> While rmlocked radix/lle (and ifa_ref / ifa_unref, and bunch of other) changes are not there yet, you can probably get
>> x2-x4 forwarding/output performance for at least IPv4 traffic (e.g. 2-3mpps depending on test conditions).
>> 
>> In contrast, I haven't seen IPSEC being able to process more than 200kpps for any kind of workload.
>> 
>> What we've discussed with glebius@ and jmg@ at EuroBSDCon was to modify PFIL to be able to monitor/enforce
>> hooks ordering and make IPSEC code usual pfil consumer. In that case, running GENERIC with IPSEC but w/o any SA
>> won't influence packet processing path.  This also simplifies the process of making IPSEC loadable.
>> 
> 
> How soon do you think the pfil patch would be ready?  That sounds like a good first step
> towards making all this enabled in HEAD so that we can make sure that IPSec gets the broad
> testing that it needs.

I don’t think making IPsec an pfil consumer is actually feasible but that’s a different story.


> Also, if folks who know about these problems can send workloads and test descriptions to the
> list that would be very helpful.
> 
> Specific drivers and hardware types would be most helpful as this may be device related
> as well.
> 
> I will turn this on on some machines in the network test lab to see what I can see.

What you would want for the moment is a single read-mostly (read-lockless, read-non-atomic) integer that tells you if you have any policy in the system;  that’s for your branch statement.   That’s probably the closest you can get to enabling it cheaply in GENERIC without doing much.

There’s a tradeoff in that for a few packets the policy might not be effective immediately but given the amount of time it takes to “install” the policy thinking about any instant real-time guarantees here is not feasible anyway.


Just my 2cts.

Bjoern