Ignorant user overfilled /usr; strange errors followed. Fixed with fsck -y, but what exactly happened?

boink lordboink at gmail.com
Wed May 17 16:12:14 PDT 2006


Dear FreeBSD,

While attempting to follow filesystem activity (and not realising the
effect it would have) I left a ktrace running (something like ktrace
-di -p0 -ti - seen on a forum while searching for a Filemon equivalent
for FreeBSD).  I had launched it while in /usr/ports, and forgot about
it.  Doh!

Some time later strange things started happening, so I exited from
X/KDE, and found a whole sequence of 'filesystem is full' errors on
the console.  df said that /usr was at 108%.

I found the offending ktrace.out file in /usr/ports (~25GB), killed
ktrace, deleted the file, and restarted X/KDE.  It took about 5
minutes to show the desktop, following a number of console messages
like 'xauth: creating new authority file...'

So, I Googled around and found a few hints relating to permissions in
~, but that didn't seem to  be the problem; anyway I moved my ~/.kde/
to ~/.kde_20060515 in an attempt to 'reset' kde's startup behaviour,
but X/KDE still took ages to start the GUI, following similar
messages.

Even once X/KDE had started, some applications took a *long* time to
start (eg Firefox - 3/4 minutes to show blank home page).  However, I
could still r/w access all files within /usr.

Following the advice found on various fora, I then ran fsck -y in
single-user mode, with the following output:
** Last mounted on /usr
** Phase 1 - Check Blocks and Sizes
** Phase 2 - Check Pathnames
** Phase 3 - Check Connectivity
** Phase 4 - Check Reference Counts
** Phase 5 - Check Cyl groups
FREE BLK COUNT(S) WRONG IN SUPERBLK
SALVAGE? yes

401256 files, 4135860 used, 13715938 free (126762 frags, 169-647
blocks, 0,8% fragmentation)

Afterwards, X/KDE started with its usual alacrity, so I restored my
~/.kde folder, and now everything seems to be exactly as before.

My questions are these:
- How could /usr have been at 108%, given that 100% is the logical maximum?
- Is there any risk that a physically adjacent filesystem be damaged?
- How can it be that, immediately following the deletion of
/usr/ports/ktrace.out, I still had errors *at all* (prior to the
fsck)?
- Given its recent history, can I now trust the integrity of /usr?

My interpretation of the situation is that following the deletion of
/usr/ports/ktrace.out, KDE, on startup, tried to write a file to a
filesystem that apparently (although incorrectly) had no free blocks.
fsck fixed the problem of the reported free blocks.

However, if this interpretation is correct, it shouldn't have worked
*at all* until fsck had repaired the filesystem.  That is, KDE should
not have started at all.

So, prior to  the fsck -y in single-user mode, what was happening in
the five minutes it took KDE to start?

Sorry if this whole description was long and convoluted, but I'd
really appreciate a few hints as to what went on.

With thanks in advance for any illumination,
boink

PS.  Henceforth, I promise not to let commands I don't understand fill
up my filesystems :o/


More information about the freebsd-questions mailing list