Freebsd IP Forwarding performance (question, and some info) [7-stable, current, em, smp]

Andre Oppermann andre at
Mon Jul 7 09:47:06 UTC 2008


to get a systematic analysis of the performance please do the following
tests and put them into a table for easy comparison:

  1. inbound pps w/o loss with interface in monitor mode (ifconfig em0 monitor)

  2. inbound pps w/ fastforward into a single blackhole route

  3. inbound pps /w fastforward into a single blackhole route w/ ipfw and
     just one allow all rule

  4. inbound pps /w fastforward into a single blackhole route w/ ipfw and
     just one deny all rule

  5. inbound pps /w fastforward into the disc(4) discard network interface

  6. inbound pps /w fastforward into the disc(4) discard network interface
     w/ ipfw and just one allow all rule

All surrounding parameters like RX/TX interface queue length, scheduler
and so may me varied but should be noted.


Paul wrote:
> UP 32 bit test vs 64 bit:
> negligible difference in forwarding performance without polling
> slightly better polling performance but still errors at lower packet rates
> same massive hit with ipfw loaded
> Installing dragonfly in a bit..
> If anyone has a really fast PPC type system or SUN or something i'd love 
> to try it :)
> Something with a really big L1 cache :P
> Paul wrote:
>> no major differences with SMP with ULE/4BSD and preemption ON/OFF
>> 32 bit UP test coming up with new cpu
>> and I'm installing dragonfly sometime this weekend :]
>> UP: 1mpps in one direction with no firewall/no routing table is not 
>> too bad, but 1mpps both directions is the goal here
>> 700kpps with full bgp table in one direction is not too bad
>> Ipfw needs a lot of work, barely gets 500kpps with no routing table 
>> with a few ipfw rules loaded.. that's horrible
>> Linux barely takes a hit when you start loading iptables rules , but 
>> then again linux has a HUGE problem with routing
>> random packet sources/ports .. grr
>> My problem Is I need some box to do fast routing and some to do 
>> firewall.. :/
>> I'll have 32 bit 7-stable UP test with ipfw/routing table and then 
>> move on to dragonfly.
>> I'll post the dragonfly results here as well as sign up for their 
>> mailing list.
>> Bart Van Kerckhove wrote:
>>> Hash: SHA1
>>> Paul / Ingo,
>>>>> I tried all of this :/  still, 256/512 descriptors seem to work the
>>>>> best. Happy to let you log into the machine and fiddle around if you
>>>>> want :)       
>>> I've been watching this thread closely, since I'm in a very similair
>>> situation.
>>> A few questions/remarks:
>>> Does ULE provide better performance than 4BSD for forwarding?
>>> Did you try freebsd4 as well? This thread had a report about that quite
>>> opposite to my own experiences, -4 seemed to be a lot faster at 
>>> forwarding
>>> than anything else I 've tried so far.
>>> Obviously the thing I'm interested in is IMIX - and 64byte packets.
>>> Does anyone have any benchmarks for DragonFly? I asked around on IRC, 
>>> but
>>> that nor google turned up any useful results.
>>> <snip> 
>>>> I don't think you will be able to route 64byte packets at 1gbit
>>>> wirespeed (2Mpps) with a current x86 platform.
>>> Are there actual hardware related reasons this should not be 
>>> possible, or
>>> is this purely lack of dedicated work towards this goal?
>>> <snip>
>>>> Theres a "sun" used at quagga dev as bgp-route-server.
>>>> (but they don't answered my question regarding fw-performance).
>>> the Quagga guys are running a sun T1000 (niagara 1) route server - I 
>>> happen
>>> to have the machine in my racks,
>>> please let me know if you want to run some tests on it, I'm sure they 
>>> won't
>>> mind ;-)
>>> It should also make a great testbed for SMP performance testing imho 
>>> (and
>>> they're pretty cheap these days)
>>> Also, feel free to use me as a relay for your questions, they're not 
>>> always
>>> very reachable.
>>> <snap>
>>>> Perhaps you have some better luck at some different hardware systems
>>>> (ppc, mips, ..?) or use freebsd only for routing-table-updates and
>>>> special network-cards (netfpga) for real routing.
>>> The netfpga site seems more or less dead - is this project still alive?
>>> It does look like a very interesting idea, even though it's currently 
>>> quite
>>> linux-centric (and according to docs doesn't have VLAN nor ip6 
>>> support, the
>>> former being quite a dealbreaker)
>>> Paul: I'm looking forward to the C2D 32bit benchmarks (maybe throw in a
>>> freebsd4 and/or dragonfly bench if you can..) - appreciate the lots of
>>> information you are providing us :)
>>> Met vriendelijke groet / With kind regards,
>>> Bart Van Kerckhove
>>> eca31f7WQ/oXq9tJ8TEDN3CA
>>> =YGYq
>>> -----END PGP SIGNATURE-----
>> _______________________________________________
>> freebsd-net at mailing list
>> To unsubscribe, send any mail to "freebsd-net-unsubscribe at"
> _______________________________________________
> freebsd-net at mailing list
> To unsubscribe, send any mail to "freebsd-net-unsubscribe at"

More information about the freebsd-net mailing list