How to recover disk space after "filesystem full"

[Steve Bertrand]

Luke Dean wrote:
> 
> Yes, it sounds like a stupid question, but let me tell the story.
> 
> The log for my dhcp server filled up /var last night, which meant that
> dhcpd was also unable to hand out new leases, which meant that I had
> effectively been DOSed.  I'll have to look into changing my logging
> policies.
> 
> So, to correct the problem, I log into the router, removed the big
> log and several other files in /var to free up some space, and assumed
> this would correct the problem.
> 
> It did not.
> Several minutes after freeing up a lot of space on /var, I continued
> to get "filesystem full" messages and "df" continued to show the
> capacity at >100%.  I checked "df -i" for the inodes too.  That was
> fine.  I ran a quick fsck to see if that might shock the system into
> seeing all the space that I'd freed up, but no good.
> 
> I ended up rebooting the box.
> 
> Was there any other possible solution I could've tried?

You have to restart the service that was holding the log file(s) open.
The system does not release the space while an application is 'using'
the file, even after it's been deleted.

> Why wouldn't the free space immediately appear as free?

Because technically, the space is not freed. "lsof" will help identify
which process(es) are holding a particular file open, if you see that
disk space is not recovered as expected after deletion:

# pkg_add -r lsof

pearl# lsof | grep auth.log

syslogd     850     root   15w    VREG      0,127      75199  237484
/var/log/auth.log

Steve
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3233 bytes
Desc: S/MIME Cryptographic Signature
Url : http://lists.freebsd.org/pipermail/freebsd-questions/attachments/20090522/1a0dbf9a/smime.bin