OS choice for an edge router

[Gary Palmer]

Kirc Gover wrote:
> Hi Gary, Thanks for your response.
>
> Yes, the bus architecture would be either PCI-X or PCI-Express. Host CPU would be a high performance multi-core such as Xeon and NICs would be Intel.
>
> One of my concern is on the native forwarding capability of FreeBSD OS and the execution of critical userland processes. I have experience before that a FreeBSD box configured as router appears to slow down the userland processes when the traffic load is high. I have verified this lately on 6.1, running on Athlon64 with 1G NIC cards with PF and ALTQ (queuing) enabled. I'm not  so sure if this is caused by PF or ALTQ. Looking at the processes using top, it could see that "swi(x) net" process is almost as near 100% cpu utilization. At this state, the box can still forward traffic (not sure yet if the was a change in throughput) but I could notice the userland processes to be very slow, like  invoking any command from the shell (e.g. ls) will take so long to be executed and completed. Is this a know limitation or bug?
>   

I'm probably not the best person to address that, but its my 
understanding that interrupts will preempt other activity such as 
userland processing.  Since there is often a requirement to service 
interrupts as a matter of priority (e.g. before the receive buffer in 
the NIC overflows) I'm not sure theres a way around it.  The performance 
improvements in 7.x, especially relating to multi-core/CPU environments 
might help.  The complexity of the PF / ALTQ rules can also have an 
impact, although I'm a little surprised that they counted towards 
interrupt activity, which your message seems to imply.

Gary

P.S.  Please don't top post.

> Thanks a lot.
> Kirc
>
> Gary Palmer <gpalmer at freebsd.org> wrote: On Fri, Sep 07, 2007 at 03:06:54AM +1000, Kirc Gover wrote:
>   
>> We are in the stage of planning and research for a commercial development of an edge router that will be based mostly on OpenSource software. I would like to solicit for information and recommendation if FreeBSD is a suitable OS. The router is expected to withstand forwarding of sustained traffic from 10Mbps to 1Gbps and maybe more than that. Are there any known limitations of FreeBSD in terms of architecture and performance? Can I just take out a FreeBSD as is and put it with the hardware without any specific or major refinements in its code? I'm  very much concerned with its capability in forwarding heavy sustained traffic. Packet loss should be at minimum and critical userland processes should working normally  even under heavy load. Are there any known specific limitations of FreeBSD? I have browsed through the archives and found a lot of hangups, deadlocks and freeze issues. What is the usual or minimum hardware requirement? Is soekris box enough, or dual core or ASIC
>>  based platforms? I'm aware that there are so many FreeBSD based routers and network based devices in the market. Is this a way to go over realtime and embedded OS such as VxWorks and others (mostly commercial) without putting the licensing cost in picture? I really appreciate any help, suggestions and recommendations. More power to FreeBSD!
>>     
>
> Kirc,
>
> There are two factors to consider:
>
> - bus architecture (PCI, PCI-X, PCI-Express, etc) will dictate the
>   maximum throughput in bits/sec.  Allow some overhead for bus arbitration
>   activities, and remember that the packet crosses the bus twice, once
>   on the inbound leg and once on the outbound leg.
>
> - Host CPU (and perhaps to a limited extent the interface cards used)
>   will dictate the packets per second (PPS)
>
> Most commercial routers run out of packets per second (in real-world
> situations, not lab mockups) long before the theoretical maximum
> throughput is achieved. Thinks like ICMP ping packets and TCP RST
> packets are small (less than 100 bytes usually) but normally take as much
> CPU to process & route as a 1500 byte (or larger) packet.
>
> The more you put in the processing path (e.g. packet filters, complex
> routing tables) the more you reduce the PPS.
>
> Gary
>
>
>        
> ---------------------------------
> Sick of deleting your inbox? Yahoo!7 Mail has free unlimited storage. Get it now.
>