Phantom /var full messages

[Paul Schmehl]

I'm running snort 2.1.3 and mysql 3.23.58 on FreeBSD 4.9 RELEASE.  All 
applications are built from ports.

Periodically I get /var full messages and everything comes to a grinding 
halt.  The problem is, /var isn't full.

df -h will show /var at 104%, but du -h /var shows /var at 40% (for 
example).  If I shut down snort and mysql, wait for a minute and then start 
them back up, df agrees with du again.

The system works fine because only /var is full (although things can get 
squirrelly if I let it go long enough because the system can't write to the 
logs or the mail spool), so I can still ssh in and run utilities.

I suspect this is some sort of filehandle not being released issue, but I'm 
not sure how to track it down.  I've got lsof installed, but I'm not an 
expert on it yet.

Any hints would be welcomed.  What's the best way to troubleshoot this 
problem?

Paul Schmehl (pauls at utdallas.edu)
Adjunct Information Security Officer
The University of Texas at Dallas
AVIEN Founding Member
http://www.utdallas.edu