ping "No buffer space available" (freebsd 8.2)

[Sébastien Grenier]

Hi folks

We have a Sun Server X2200 M2 machine[1] running FreeBSD 8.2, and we are
having serious reliability issues with the network routing. This router
used to be another machine (a generic PC) running OpenBSD 4.2, and was
running perfectly fine.

After a while (a few hours to a few days), we get the following error
message when trying to ping any IP:  "No buffer space available".

After doing some research, we have tried to do some tuning of the
following kernel parameters:

sysctl -w net.inet.tcp.sendbuf_max = 16777216
sysctl -w net.inet.tcp.recvbuf_max = 16777216
sysctl -w kern.ipc.maxsockbuf=400000
sysctl -w net.inet.tcp.recvspace=512000

We still get the error message "No buffer space available" after a while.

So far, the only wait to get around the problem is to reboot the machine
and reapply the above configuration, which keeps the problem away for a
few days instead of a few hours.

We've enabled the collection of statistics on the router with munin[2]

We didn't apply the above configurations in /boot/loader.conf because we
still experience the error...

Any idea?  What would be a good way to debug? Are they values that we
should be monitoring?

Regards,

Sébastien
Koumbit.org

[1] more details here: https://wiki.koumbit.net/rtr0.koumbit.net

[2] http://stats.koumbit.net/koumbit.net/rtr0.koumbit.net/index.html


(when it's routing ok.)
rtr0# netstat -m
829/2246/3075 mbufs in use (current/cache/total)
799/1861/2660/51200 mbuf clusters in use (current/cache/total/max)
798/1378 mbuf+clusters out of packet secondary zone in use (current/cache)
0/35/35/25600 4k (page size) jumbo clusters in use (current/cache/total/max)
0/0/0/12800 9k jumbo clusters in use (current/cache/total/max)
0/0/0/6400 16k jumbo clusters in use (current/cache/total/max)
1807K/4423K/6230K bytes allocated to network (current/cache/total)
0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0/0/0 requests for jumbo clusters denied (4k/9k/16k)
0/0/0 sfbufs in use (current/peak/max)
0 requests for sfbufs denied
0 requests for sfbufs delayed
0 requests for I/O initiated by sendfile
0 calls to protocol drain routines


-- 

L'éducation n'est pas une marchandise,
L'éducation est un droit
Luttons pour ce droit
http://www.youtube.com/watch?v=ek2B3Nd7Ofk