igb/ixgbe RSS/RX queues for non-IP traffic

Alexander V. Chernikov melifaro at yandex-team.ru
Wed Sep 7 14:53:06 UTC 2011


Hello list!

(CC'in Jack Vogel since this is intel drivers/hardware specific question)

There are some techniques for assigning network traffic to different NIC 
RX queues (bound to different CPUs/cores).
The main problem for all techniques is avoiding any possible packet 
reordering in single flow.

Most obvious (and most used) are described in Microsoft NDIS driver 
specification:
Hashing is done on various tuples like:
* IPv46 src/dst + TCP/UDP src/dst port for TCP/UDP
* IPv46 src/dst for general IPv46 traffic

igb's have a bit more options like assigning selected ethertype or TCP 
SYN to different queue (See section 7.1.1 of 82576EB datasheet)
ixgbe (82599) goes even more far: flow director functionality permits 8K 
'perfect match' filters allowing to select any subset in:
* vlan
* IPv46 src/dst
* L4 proto
* TCP/UDP/SCTP protocol ports
and even 'flexible 2-byte tuple anywhere in the first 64 bytes of the 
packet'

There are other possibilities to assign traffic to different queues in 
ixbge like using .1q priority filed (DCB functionality)
More information can be found in sections 7.1.2 (RX queues assignment), 
7.1.2.7(Flow Director), 7.7.1 (DCB) of 82599 datasheet

However, there are many setups where PC can be used as platform for 
routing/dispatching non-IP traffic.
PPPoE server is a typical example.

People have to do some tricks (link aggregation, non-direct ISR) to get 
traffic dispatched by more than single CPU/core, but this much less 
efficient.
I'm a bit curious: why some generic hashing mechanism based on something 
like 'flexible 2-byte tuple anywhere in the first 64 bytes of the 
packet' were not added?
It can be (at least from my point of view) easily based on (for example) 
flow director functionality.

Particularly I'm trying to figure out how can I use all this variety of 
filters to get MPLS traffic split to different RX queues.

Maybe someone can point me to the right direction?

-- 
Alexander V. Chernikov
Yandex NOC


More information about the freebsd-net mailing list