/ file system is full, but du does not show that it's full

[c0re]

2011/2/28 Robert Bonomi <bonomi at mail.r-bonomi.com>:
>> From owner-freebsd-questions at freebsd.org  Mon Feb 28 05:31:46 2011
>> Date: Mon, 28 Feb 2011 14:24:30 +0300
>> From: c0re <nr1c0re at gmail.com>
>> To: Matthew Seaman <m.seaman at infracaninophile.co.uk>
>> Cc: FreeBSD <freebsd-questions at freebsd.org>
>> Subject: Re: / file system is full, but du does not show that it's full
>>
>> 2011/1/6 Matthew Seaman <m.seaman at infracaninophile.co.uk>:
>> > On 06/01/2011 11:26, c0re wrote:
>> >> # df -h
>> >> Filesystem     Size    Used   Avail Capacity  Mounted on
>> >> /dev/ad0s1a    496M    466M   -9.8M   102%    /
>> >>
>> >> So it's full.
>> >>
>> >> But by du it's not appeared to be full
>> >>
>> >>
>> >> # du -hxd 1 /
>> >> 2.0K    /.snap
>> >> 512B    /dev
>> >> 2.0K    /tmp
>> >> 2.0K    /usr
>> >> 2.0K    /var
>> >> 1.9M    /etc
>> >> 2.0K    /cdrom
>> >> 2.0K    /dist
>> >> 1.0M    /bin
>> >> 131M    /boot
>> >>  10M    /lib
>> >> 356K    /libexec
>> >> 2.0K    /media
>> >>  12K    /mnt
>> >> 2.0K    /proc
>> >> 7.2M    /rescue
>> >> 296K    /root
>> >> 4.7M    /sbin
>> >> 4.0K    /lost+found
>> >> 157M    /
>> >>
>> >
>> > Do you have partitions mounted at /tmp, /usr, /var etc?  Does the
>> > output of your du command change if you unmount those partitions? (It
>> > might be an idea to boot into a livefs CD or DVD given that du(1) lives
>> > in /usr/bin, so a bit tricky to unmount /usr and then run du)
>> >
>> > My guess is that you've at one time created files beneath what is
>> > usually a mount point.  Mounting the partition over them makes those
>> > files inaccessible, but they still take up space on the drive.
>> >
>> >        Cheers,
>> >
>> >        Matthew
>> >
>> > --
>> > Dr Matthew J Seaman MA, D.Phil.                   7 Priory Courtyard
>> >                                                  Flat 3
>> > PGP: http://www.infracaninophile.co.uk/pgpkey     Ramsgate JID:
>> > matthew at infracaninophile.co.uk               Kent, CT11 9PW
>> >
>> >
>>
>> At last I found time to check it. Booted with frenzy life cd, mounted
>> only / partition and saw trash
>> /var/spool. Deleted it and it solved problem.
>> But later was and idea to mount device of / (/dev/da0s1a) as /mnt/root
>> and just delete those files without need of livecd. It works in Linux.
>> But in freebsd i got
>>
>> # mount /dev/da0s1a /mnt/root/
>> mount: /dev/da0s1a : Operation not permitted
>>
>> So only single user mode or live cd could solve it.
>
> *NOT* true.  Stopping any daemons that were using "/var/spooll", and then
> umount(1)-ing it would have done the trick from multi-user mode.
>

Yeah, not true.

Checked with lsof /var and it was used by these daemons:

devd
syslogd
rpcbind
snmpd
mysqld
httpd
sendmail
cron

Yes, I can stop them all,  but was not sure about stopping devd...