var Filesystem Full Help

[Sean Murphy]

Derek Ragona wrote:
> I would reboot and run fsck in single user mode.  The reboot will 
> clear any old open files that may be causing the drive full problem.
>
>         -Derek
>
>
> At 11:59 AM 4/3/2007, Sean Murphy wrote:
>> Sean Murphy wrote:
>>> I am getting these errors on my var filesystem but df -h shows there 
>>> is plenty of space available.
>>>
>>> I am running FreeBSD 5.4
>>>
>>> muse2# df -h
>>> Filesystem       Size    Used   Avail Capacity  Mounted on
>>> /dev/amrd0s1a    989M     56M    854M     6%    /
>>> devfs            1.0K    1.0K      0B   100%    /dev
>>> /dev/amrd0s1e    989M     32M    878M     4%    /tmp
>>> /dev/amrd0s1f    9.5G    4.2G    4.6G    48%    /usr
>>> /dev/amrd0s1g    245G    9.4G    216G     4%    /usr/home
>>> /dev/amrd0s1d    1.9G    526M    1.3G    29%    /var
>>>
>>> muse2# tail /var/log/messages
>>> Apr  3 09:00:44 muse2 kernel: pid 537 (mimedefang), uid 26 inumber 
>>> 126291 on /var: filesystem full
>>> Apr  3 09:09:55 muse2 kernel: pid 52000 (httpd), uid 80 inumber 
>>> 170037 on /var: filesystem full
>>> Apr  3 09:12:59 muse2 kernel: pid 34758 (mimedefang), uid 26 inumber 
>>> 127701 on /var: filesystem full
>>>
>>> I have restarted the mimdefang process but I get the same messages.
>>>
>>> What can I do?
>>> _______________________________________________
>>> freebsd-questions at freebsd.org mailing list
>>> http://lists.freebsd.org/mailman/listinfo/freebsd-questions
>>> To unsubscribe, send any mail to 
>>> "freebsd-questions-unsubscribe at freebsd.org"
>>
>>> Purge some old log files to start with. However according to your df 
>>> above you should have plenty of room
>> I have deleted logs and lost+found but the same messages still show up.
>>
>>> Check df -i as you may have run out of inodes rather than out of 
>>> file space.
>>
>> Looks as though I have plenty of inodes
>>
>> muse2# df -i
>> Filesystem    1K-blocks    Used     Avail Capacity iused    ifree 
>> %iused  Mounted on
>> /dev/amrd0s1a   1012974   57694    874244     6%    1520   139790    
>> 1%   /
>> devfs                 1       1         0   100%       0        0
>> 100%   /dev
>> /dev/amrd0s1e   1012974   33192    898746     4%   11141   130169
>> 8%   /tmp
>> /dev/amrd0s1f   9938894 4368142   4775642    48%  279371  1015987
>> 22%   /usr
>> /dev/amrd0s1g 257098734 9794488 226736348     4%  113153 33118717
>> 0%   /usr/home
>> /dev/amrd0s1d   2026030  510498   1353450    27%     866   281756
>> 0%   /var
>> muse2#
>>
>>> Have you rebooted since the
>>> problem showed up?
>> I have not rebooted yet
>>
>>
>> can I fun fsck on a live filesystem or do I have to drop into single 
>> usermode?
>> _______________________________________________
>> freebsd-questions at freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-questions
>> To unsubscribe, send any mail to 
>> "freebsd-questions-unsubscribe at freebsd.org"
>>
>> -- 
>> This message has been scanned for viruses and
>> dangerous content by MailScanner, and is
>> believed to be clean.
>> MailScanner thanks transtec Computers for their support.
>>
>


I ran the # fstat -f /var and compared it to the # find /var -inum nnnnn 
-print command.  I found mysqld httpd mimedefang syslogd all had 
processes open but not written the files to /var.  I ended up rebooting 
into single user mode and running fsck -f -y /var twice just to make 
sure it was clean.  There were many errors on /var reported and fixed by 
fsck.

here is my new output of df -h

muse2# df -h
Filesystem       Size    Used   Avail Capacity  Mounted on
/dev/amrd0s1a    989M     56M    854M     6%    /
devfs            1.0K    1.0K      0B   100%    /dev
/dev/amrd0s1e    989M     23M    888M     2%    /tmp
/dev/amrd0s1f    9.5G    4.2G    4.5G    48%    /usr
/dev/amrd0s1g    245G    9.3G    216G     4%    /usr/home
/dev/amrd0s1d    1.9G    357M    1.4G    20%    /var
muse2#

/var is smaller now it was

/dev/amrd0s1d    1.9G    526M    1.3G    29%    /var

also the current inode count

muse2# df -i
Filesystem    1K-blocks    Used     Avail Capacity iused    ifree 
%iused  Mounted on
/dev/amrd0s1a   1012974   57762    874176     6%    1524   139786    1%   /
devfs                 1       1         0   100%       0        0  
100%   /dev
/dev/amrd0s1e   1012974   23064    908874     2%   11481   129829    
8%   /tmp
/dev/amrd0s1f   9938894 4376480   4767304    48%  279523  1015835   
22%   /usr
/dev/amrd0s1g 257098734 9763632 226767204     4%  114302 33117568    
0%   /usr/home
/dev/amrd0s1d   2026030  365632   1498316    20%   12198   270424    
4%   /var
muse2#

it was

/dev/amrd0s1d   2026030  510498   1353450    27%     866   281756    
0%   /var

the 0% is interesting why would it report 0% when it was having problems?


Thanks for all your help and advice.  The system seems stable now.