resource leak

[Robert Watson]

On Wed, 1 Oct 2008, Stephen Clark wrote:

> A big part of problem is this seems to take about 100 days of uptime to 
> occur. We have some inhouse test boxes but have never seen the problem, 
> probably because non of them have been up more than about 45 days. The units 
> in the field, of which there is about 300, are headless and none are 
> physically close.
>
> When the boxes are rebooted there are no error messages in any of the log 
> files, only the absence of information that would normally be logged by new 
> processes that would be spawned. We are getting ready to install a patch 
> that will try to gather more information.
>
> I thought about writing an app the would try to fork a child periodically 
> and record in a log file if there was an error. But EAGAIN is nonspecific as 
> to the real reason the fork failed. I was looking for some way to 
> periodically log the resources that would cause the fork failure.

The narrowness of the UNIX errno space is, at times, fairly unhelpful.

As far as I'm aware, the two main causes of EAGAIN out of fork() are an 
exhaustion of maxprocs or an exhaustion of per-user process limits.  This 
suggests one or more run-away applications or services, or a gradual leak of 
processes from a service (perhaps a failure to GC dead children, or a gradual 
increase but never decrease of worker processes?).

Robert N M Watson
Computer Laboratory
University of Cambridge

>
> procstat -k looks like it would have been a good candidate but unfortunately 
> we
> are running 6.1.
>
> Thanks for the response.
> Steve
>
> -- 
>
> "They that give up essential liberty to obtain temporary safety,
> deserve neither liberty nor safety."  (Ben Franklin)
>
> "The course of history shows that as a government grows, liberty
> decreases."  (Thomas Jefferson)
>
>
>