Major problem with "No buffer space available" errors

[Terry Lambert]

James West wrote:
> I'm having huge problems with "No buffer space available" errors. I've
> increased MAXUSERS to 512 in the kernel, recompiled, rebooted and the
> sysctl values below show that everything is up'ed to the max.

Historically, the most common cause of this message has been an
interface you were actively feeding data going down.  Most commonly,
this occurs in the use of UDP and ICMP protocols (e.g. "ping") out a
downed interface, since they are not constrained by your sendspace
settings, which apply to TCP (they will basically permit you to use
all your buffer in the attempt).

So the questions to answer are:

1)	Are you using a lot of ICMP (e.g. ping, traceroute, RIP, etc.)?

2)	Are you using a lot of UDP (e.g. Linux NFS clients using UDP
	mounts and an rsize or wsize larger than the MTU would permit
	to fit in a single UDP packet)?

Other than that, you should "netstat -an" and add up the contents of
the SendQ/RecvQ columns.  It's possible that, in fact, you *are*
running out of buffer space.  For the default of 64K, you would need
2G of RAM dedicated to nothing but mbufs (not including headers!) to
support only 32,768 simultaneous connections without mbuf overcommit.

Dropping the sndspace/rcvspace paramaters back to their pre-bump-up
defaults will double the number of connections for the same amount
of RAM.

-- Terry