Handling 100.000 packets/sec or more

["." at babolo.ru]

I administer some home networks with 200..500 users on port
and 5..12 ports on each router.

The trouble is that router can't do somethig useful when
link saturated. The only effective way found is
2..3 mb/s restriction _from_ every user on each switch port

PS
typical router has Tyan 2466N-4M mobo with one Athlon XP 2400+
512M (a lot of pipes) and FreeBSD 4 STABLE.
DragonFlyBSD looks good but I haven't any in production yet.

FreeBSD 5 is not production quality (last test about one mounts ago).

>   At one site that I administer we have a gateway server which services
> a large SOHO LAN (more than 300 stations) and I'm facing a serious
> issue: very often we see strong spoofed floods (variable source IP and
> port, variable destination IP, destination port 80) which can go as far
> as 100 000 packets/sec!
> 
>   Of course, the server (FreeBSD 5.2-REL, PIII 733Mhz, 256Mb RAM, 3COM
> 3C905B-TX aka xl0 with checksum offloading support) has a hard time
> swallowing this kind of traffic. The main issue are the IRQ interrupts:
> over 15000 interrupts/sec which consume more than 90% of the CPU time.
> We got ingress filtering so the packets go no further than the firewall
> (which, BTW, is not the issue, even disabling it it's the same problem).
> The system is still responsive but the load average goes as high as 10
> and the interface is losing packets (input errors) which dramatically
> affects legitimate traffic, besides mbuf(9) starvation. We are taking
> down the culprit clients, but this takes time and we need the other
> clients not to be affected by it.
> 
>   What can I do to make the system better handle this kind of traffic ?
> Could device polling(8) or just increasing the kernel frequency clock to
> 1000Hz or more improve the situation ?
>   What kind of network cards could face a lot better this burden ? Are
> there any other solutions ?
> 
>   On a side note: what would be a adequate formula to calculate the
> NMBCLUSTERS and MBUFS we should set on this server (via boot-time
> kern.ipc.nmbclusters and kern.ipc.nmbufs) ?
> 
> -- 
> Adrian Penisoara
> Ady (@freebsd.ady.ro)