Socket leak (Was: Re: What triggers "No Buffer Space) Available"?

[Kris Kennaway]

On Mon, May 07, 2007 at 07:01:02PM +0200, Oliver Fromme wrote:
> Marc G. Fournier wrote:
>  > Now, that makes sense to me, I can understand that ... but, how would
>  > that look as far as netstat -nA shows?  Or, would it?  For example, I
>  > have:
> 
> You should use "-na" to list all sockets, not "-nA".
> 
>  > mars# netstat -nA | grep c9655a20
>  > c9655a20 stream      0      0        0 c95d63f0        0        0
>  > c95d63f0 stream      0      0        0 c9655a20        0        0
>  > mars# netstat -nA | grep c95d63f0
>  > c9655a20 stream      0      0        0 c95d63f0        0        0
>  > c95d63f0 stream      0      0        0 c9655a20        0        0
>  > 
>  > They are attached to each other, but there appears to be no 'referencing 
>  > process'
> 
> netstat doesn't show processes at all (sockstat, fstat
> and lsof list sockets by processes).  The sockets above
> are probably from a socketpair(2) or a pipe (which is
> implemented with socketpair(2), AFAIK).  That's perfectly
> normal.
> 
> If I remember correctly, you wrote that 11k sockets are
> in use with 90 jails.  That's about 120 sockets per jail,
> which isn't out of the ordinary.  Of course it depends on
> what is running in those jails, but my guess is that you
> just need to increase the limit on the number of sockets
> (i.e. kern.ipc.maxsockets).

Yes, and if you have 11000 sockets in use under "normal" situations
then you're likely to be pressing right up against the default limit
anyway (e.g. on this machine with 8GB of RAM the default is 12328), so
a slight increase in load will run out of space.

Kris