Routing SMP benefit
fazaeli at sepehrs.com
Thu Jan 3 01:48:05 PST 2008
Where are 'those numbers' you are referring to? have I missed some message
in the thread?
Tiffany Snyder wrote:
> Hi Andre,
> are those numbers for small (64 bytes) packets? Good job on pushing
> the base numbers higher on the same HW.
> What piqued my attention was the note that our forwarding
> performance doesn't scale with multiple CPUs. Which means there's a lot of
> work to be done :-) Have we taken a look at OpenSolaris' Surya
> project? They allow multiple readers/single writer on the radix_node_head
> (and not a mutex as we do) and we may be able to do the same to gain some
> parallelism. There are other things in Surya that exploit multiple CPUs.
> It's definitely worth a read. DragonFlyBSD seems to achieve parallelism by
> classifying packet as flows and then redirecting the flows to different
> CPUs. OpenSolaris also does something similar. We can definitely think along
> those lines.
> 1) I said multiple instead of dual CPUs on purpose.
> 2) I mentioned OpenSolaris and DragonFlyBSD as examples and to acknowledge
> the work they are doing and to show that FreeBSD is far behind and is losing
> it's lustre on continuing to be the networking platform of choice.
> On 12/29/05, Andre Oppermann <andre at freebsd.org > wrote:
>> Markus Oestreicher wrote:
>>> Currently running a few routers on 5-STABLE I have read the
>>> recent changes in the network stack with interest.
>> You should run 6.0R. It contains many improvements over 5-STABLE.
>>> A few questions come to my mind:
>>> - Can a machine that mainly routes packets between two em(4)
>>> interfaces benefit from a second CPU and SMP kernel? Can both
>>> CPUs process packets from the same interface in parallel?
>> My testing has shown that a machine can benefit from it but not
>> much in the forwarding performance. The main benefit is the
>> prevention of lifelock if you have very high packet loads. The
>> second CPU on SMP keeps on doing all userland tasks and running
>> routing protocols. Otherwise your BGP sessions or OSPF hellos
>> would stop and remove you from the routing cloud.
>>> - From reading the lists it appears that net.isr.direct
>>> and net.ip.fastforwarding are doing similar things. Should
>>> they be used together or rather not?
>> net.inet.ip.fastforwarding has precedence over net.isr.direct and
>> enabling both at the same doesn't gain you anything. Fastforwarding
>> is about 30% faster than all other methods available, including
>> polling. On my test machine with two em(4) and an AMD Opteron 852
>> (2.6GHz) I can route 580'000 pps with zero packet loss on -CURRENT.
>> An upcoming optimization that will go into -CURRENT in the next
>> few days pushes that to 714'000 pps. Futher optimizations are
>> underway to make a stock kernel do close to or above 1'000'000 pps
>> on the same hardware.
>> freebsd-net at freebsd.org mailing list
>> To unsubscribe, send any mail to " freebsd-net-unsubscribe at freebsd.org"
> freebsd-net at freebsd.org mailing list
> To unsubscribe, send any mail to "freebsd-net-unsubscribe at freebsd.org"
With best regards.
Hooman Fazaeli <fazaeli at sepehrs.com>
Sepehr S. T. Co. Ltd.
More information about the freebsd-net