Performance/lockup issues with FreeBSD as a router

[Benjamin Krueger]

* Peter Kieser (pfak at telus.net) [050526 10:49]:
> Hello guys,
> 
> I'm not quite sure if this is the right list to address this to, as it's partly
> a performance problem and partly otherwise.
> 
> I have a FreeBSD machine acting as a router (doing approx. 15-25Mbit/s of
> traffic (lot's of small packets, about 45,000 pps)), however I'm currently
> running into issues where one, or both of the NICs will stop transmitting
> traffic. When I go onto the machine, and try to ping something I get "No buffer
> space available"
> 
> The nics are if_dc, this is a stock FreeBSD 5.4-RELEASE installation (no
> firewall or anything):
> 
> Here's my /etc/sysctl.conf:
> 
> net.inet.ip.rtexpire=1800
> net.inet.ip.rtminexpire=1800
> kern.maxfiles=32768
> kern.maxfilesperproc=32768
> kern.ipc.somaxconn=32767
> net.inet.tcp.sendspace=256000
> net.inet.tcp.recvspace=256000
> kern.ipc.maxsockbuf=2097152
> net.inet.ip.fastforwarding=1
> net.inet.tcp.rfc1323=1
> net.link.ether.inet.max_age=600
> net.inet.tcp.msl=7500
> net.inet.ip.fw.dyn_udp_lifetime=10
> net.inet.ip.fw.dyn_buckets=1024
> 
> And here's my /boot/loader.conf:
> 
> kern.ipc.maxsockets="163840"
> kern.maxusers="2048"
> 
> Is there anythign I'm overlooking that would be causing the machine to lockup
> like this?
> 
> --Peter

What does 'netstat -m' say? Is there any reason you explicitely set maxusers instead of
letting the kernel auto-tune the setting? I would watch your mbuf usage and when you
find your average and peak usage, configure your mbuf allocation at boot and see if that
helps. I'd also consider letting FreeBSD auto-tune your MAXUSERS.

-- 
Benjamin Krueger
SysAdmin, CarDomain Network
92 Toyota Turbo MR2
78 Datsun B-210
91 freakin Geo Prizm