10gbps scalability (was: Re: FreeBSD problems and preliminary ways to solve)

[Robert N. M. Watson]

On 20 Aug 2011, at 15:10, Poul-Henning Kamp wrote:

> In message <alpine.BSF.2.00.1108201234280.4529 at fledge.watson.org>, Robert Watso
> n writes:
> 
>> Part of the key here will be reworking things like ipfw(4) 
> 
> Here is how to do it:
> 
> Compile IPFW rules to C-code, compile C-code to KLD, load KLD and hook
> the firewall rules.
> 
> If the C-code is designed smartly, the C-compiler can optimize to
> insanely efficient code.
> 
> The same semantics as today can be preserved with respect to counters
> and dynamic addition/removal of rules, with a little bit of creative
> thinking about data structures.
> 
> Somebody[tm] did that long ago, but never contributed the patches back
> once The Mgt[tm] realized what performance we were talking about.


I'm actually slightly less concerned about this aspect of it, although some sort of JIT/etc, perhaps grounded in LLVM, would make sense. I'm more concerned with the management of firewall state in the presence of multiple network queues and SMP. We should be able to build substantially on the approaches we've been using higher in the network stack to align NIC-level work distribution with network stack processing and application process affinity. (These ideas are still coming to maturity, but there's useful stuff to be found there.)

Robert