802.1Q vlan performance.
if at hetzner.co.za
Fri Aug 25 07:37:50 UTC 2006
While doing some experimentation and work on ipfw to see where I
could improve performance for our virtualised firewall I came across
the following comment in sys/net/if_vlan.c:
* The VLAN_ARRAY substitutes the dynamic hash with a static array
* with 4096 entries. In theory this can give a boots(sic) in processing,
* however on practice it does not. Probably this is because array
* is too big to fit into CPU cache.
Being curious and having determined the main throughput bottleneck
to be the vlan driver, I thought that I'd test the assertion. I
have have 506 vlans on this machine.
With VLAN_ARRAY unset, ipfw disabled, fastforwarding enabled,
vlanhwtag enabled on the interface, the fastest forwarding rate I
could get was 278kpps (This was a steady decrease from 440kpps with
24 vlans linearly proportional to the number of vlans).
With exactly the same configuration, but the vlan driver compiled
with VLAN_ARRAY defined, the forwarding rate of the system is back
The testbed looks like this:
|pkt gen | | router | | pkt rec |
| host |vlan2 vlan2 | |vlan1002 vlan1002 | host |
|netperf |----------->| |------------------->| netserver|
| |em0 em0 | |em1 em0 | |
The router has vlan2 to vlan264 and vlan1002 through vlan1264 in
22 blocks of 23 vlan groups (a consequence of 24 port switches to
to tag/untag for customers). The pkt gen and recieve host both
have 253 vlans.
Can anyone suggest a good reason not to turn this option on by
default. It looks to me like it dramatically improves performance.
More information about the freebsd-current