Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp]

[Kris Kennaway]

Andre Oppermann wrote:
> Robert Watson wrote:
>> Experience suggests that forwarding workloads see significant lock 
>> contention in the routing and transmit queue code.  The former needs 
>> some kernel hacking to address in order to improve parallelism for 
>> routing lookups.  The latter is harder to address given the hardware 
>> you're using: modern 10gbps cards frequently offer multiple transmit 
>> queues that can be used independently (which our cxgb driver 
>> supports), but 1gbps cards generally don't.
> 
> Actually the routing code is not contended.  The workload in router
> is mostly serialized without much opportunity for contention.  With
> many interfaces and any-to-any traffic patterns it may get some
> contention.  The locking overhead per packet is always there and has
> some impact though.
> 

Actually contention from route locking is a major bottleneck even on 
packet generation from multiple CPUs on a single host.  It is becoming 
increasingly necessary that someone look into fixing this.

Kris