Phantom /var full messages

[Bill Moran]

Paul Schmehl <pauls at utdallas.edu> wrote:
> --On Thursday, September 09, 2004 01:03:33 PM -0400 Bill Moran 
> <wmoran at potentialtech.com> wrote:
> >>
> >> Any hints would be welcomed.  What's the best way to troubleshoot this
> >> problem?
> >
> > First, if you could isolate it to just snort or just MySQL.
> >
> > Typically, folks have this problem because they try to rotate log files
> > without restarting the program that's logging to them.  The rotate program
> > compresses the current log file into a new file, then deletes the original
> > file ... but the program is still logging to it.  Thus the space fills up,
> > but there is no file to see the space in.  Restarting the program doing
> > the logging causes the old file to disappear, and a new log file to be
> > created.
> >
> > On a guess, Snort would be the first thing I'd look at.  However, MySQL
> > can create a TON of data if logging is enabled, so you may want to look
> > closely at it as well.
> >
> Thanks, Bill.  That's really helpful.  I suspected it was snort, but I 
> wasn't sure.  I'll shut down one process at a time and see when df "returns 
> to normal".  I am using newsyslog.conf which *should* HUP processes when 
> logs are turned over, but maybe I missed something.

Double-check your newsyslog config, and make sure that HUP actually does
what you think it does (not all programs support the HUP convention).

Keep in mind that it might be other things as well.  Some processes will
open a file and then delete it in order to have anonymous scratch space.
Once you've determined whether it's snort or MySQL, you may have to do
a little more detailed research into that particular program.  As a
possible scenerio, snort might expect to have a lot of disk avaiable for
scratch space, and you may have to use a command-line switch when starting
snort to tell it to use a different partition for it's scratch space.  I'm
no snort expert, though, so I'm just guessing.

Good luck, and post your solution back to the list once you've figured it
out, for the archives.

-- 
Bill Moran
Potential Technologies
http://www.potentialtech.com