Strange case of filesystem corruption?

Robert Watson rwatson at FreeBSD.org
Tue Sep 6 03:51:04 PDT 2005


On Mon, 5 Sep 2005, Mikhail Teterin wrote:

> Can this be explained by anything other than a (nasty) bug?
>
> 	% ls -la audio/shorten/files
> 	total 0
> 	% rmdir audio/shorten/files
> 	rmdir: audio/shorten/files: Directory not empty
>
> This is on 5.4-stable from July 21 -- up ever since... Thanks!

Mikhail,

Have you recently experienced a system crash or hard reboot without proper 
shutdown?  I know of at least one case prior to 6.x where this can occur 
-- a bug I reported to Kirk relating to bgfsck.  Basically, soft update 
guarantees that upon reboot after failure (power or otherwise), the 
on-disk layout of UFS meta-data will be consistent, except with respect to 
reference and link counts, which may be elevated.  What bgfsck does is 
walk the file system to identify and correct the elevated counts.  Here's 
a specific example of such a problem:

   Directory A is created
 	Write A directory data  (., ..)
 	Write A directory inode (new inode)
 	Write A parent inode    (link count++)
 	Write A parent data     (add name)
   Directory B is created
 	Write B directory data  (., ..)
 	Write B directory inode (new inode)
 	Write A directory inode (link count++)
 	XXX
 	Write A directory data  (add name)
   Directory B is removed
 	Write A directory data  (remove name)
 	XXX
 	Write A directory inode (link count--)

Note that if the sequence of events his halted at either of the XXX's 
above, the inode link count on directory A will be elevated, even though 
the name for B has been removed from A.  Background fsck comes alone 
later, notices that the counts are elevated, and drops them.  However, 
until ufs_vnops.c:1.269, this caused a problem: because the link count was 
elevated, UFS assumed that the directory contained a reference to another 
directory, and would not let it be removed.  Once bgfsck catches up with 
the directory, it can be removed.

I've seen this symptom most frequently following a crash or a power outage 
during an rm -Rf of a /usr/obj, which I then immediately restart on 
reboot, and rm -Rf gets there before bgfsck does.  Here's the commit 
message for ufs_vnops.c:1.269, which should be MFC'd:

   revision 1.269
   date: 2005/05/18 22:18:21;  author: mckusick;  state: Exp;  lines: +2 -3
   Allow removal of empty directories with high link counts. These can
   occur on a filesystem running with soft updates after a crash and
   before a background fsck has been run. To prevent discrepancies
   from arising in a background fsck that may already be running,
   the directory is removed but its inode is not freed and is left
   with the residual reference count. When encountered by the
   background fsck it will be reclaimed.

I'll e-mail Kirk and ask if he's comfortable enough with the change to 
this point to merge it.

Robert N M Watson


More information about the freebsd-questions mailing list