A bad lost+found causes fsck of FFS to repeatefly fail.

Julian Stacey jhs at berklix.org
Tue Mar 1 23:45:29 GMT 2005


Folk on freebsd-fs,

I had a FFS error that fsck couldnt fix, & I've seen similar before:
  If content of lost+found/ is itself damaged, fsck fails, it seems.
  Then one has to manually [copy elsewhere] & rm -rf lost+found then re-fsck.
Should the above be added to man fsck, or
content below submitted as a send-pr ? Or do readers here
include FFS / FSCK maintainers ?

Log & environ follow:

uname -a
  FreeBSD fire.jhs.private 5.3-RELEASE FreeBSD 5.3-RELEASE #0: Sat
  Feb 26 21:47:28 CET 2005
  jhs at fire.jhs.private:/usr1/src/sys/amd64/compile/FIRE64.small amd64

fsck: # (from later to give a feel of size of machine)
  Filesystem   Size    Used   Avail Capacity  Mounted on
  /dev/ad0s2a  260M     61M    178M    25%    /
  /dev/ad0s2d  260M     35M    204M    15%    /var
  /dev/ad0s2e  260M     80k    239M     0%    /tmp
  /dev/ad0s2f  1.2G    876M    266M    77%    /usr
  /dev/ad0s1a  260M     62M    177M    26%    /a
  /dev/ad0s1d  260M     50M    189M    21%    /a/var
  /dev/ad0s1f  1.2G    899M    248M    78%    /a/usr
  /dev/ad0s3d   53G     42G    7.0G    86%    /usr1

fstab:	(for the problematic 40 Gig partition)
  /dev/ad0s3d             /mnt            ufs     rw              2       2

fsck -y /dev/ad0s3d
  ** /dev/ad0s3d
  ** Last Mounted on /usr1
  ** Phase 1 - Check Blocks and Sizes
  ** Phase 2 - Check Pathnames
  MISSING '.'  I=5393708  OWNER=root MODE=40755
  SIZE=512 MTIME=Mar  1 12:50 2005
  DIR=/lost+found/#5393708
  
  UNEXPECTED SOFT UPDATE INCONSISTENCY
  CANNOT FIX, FIRST ENTRY IN DIRECTORY CONTAINS setup.ins
  
  UNEXPECTED SOFT UPDATE INCONSISTENCY
  ** Phase 3 - Check Connectivity
  ** Phase 4 - Check Reference Counts
  ** Phase 5 - Check Cyl groups
  1436147 files, 20432492 used, 5476106 free (107674 frags, 671054 blocks, 0.4% fr
  agmentation)

mount -r /dev/ad0s3d /mnt
cd /mnt/lost+found ; find . -type f | sort | xargs ls -l
  -rwxr-xr-x  1 jhs   staff   3333 Jan  6  2003 ./#5393695/readme.txt
  -rwxr-xr-x  1 jhs   staff  44069 Jul 27  2002 ./#5393698/camsetui.chm
  -rwxr-xr-x  1 jhs   staff  64031 Dec 12  2002 ./#5393698/fvu1.chm
  -rwxr-xr-x  1 jhs   staff   4708 Feb  5  2003 ./#5393698/readme.txt
  -rwxr-xr-x  1 jhs   staff  86165 Feb 19  2003 ./#5393703/demo.dbd
  -rwxr-xr-x  1 jhs   staff   1273 Feb  6  2003 ./#5393703/demo.txt
  -rwxr-xr-x  1 jhs   staff  74235 Mar  1 12:50 ./#5393708/setup.ins
  -rwxr-xr-x  1 jhs   staff     41 Jul  1  2002 ./#5393709
  -rwxr-xr-x  1 jhs   staff   4557 Jul  1  2002 ./#5393710
  -rwxr-xr-x  1 jhs   staff    396 Jul  1  2002 ./#5393711
  -rwxr-xr-x  1 jhs   staff    417 Jul  1  2002 ./#5393712
  -rwxr-xr-x  1 root  wheel   8192 Mar  1 12:46 ./#5393713
  -rw-r--r--  1 jhs   staff   4397 Oct  5  2002 ./#5417187
  -rw-r--r--  1 jhs   staff   6612 Oct  3  2002 ./#5417188
  -rw-r--r--  1 root  wheel      0 Mar  1 13:25 ./#5417189
  -r--r--r--  1 root  wheel   8155 May 22  2004 ./#6432678
  -r--r--r--  1 root  wheel   2333 Jan 11 04:13 ./#6432683
  -r--r--r--  1 root  wheel   2333 Jan 11 04:13 ./#6432684
  -r--r--r--  1 root  wheel  21319 Jan 11 04:13 ./#6432685
  -r--r--r--  1 root  wheel  21319 Jan 11 04:13 ./#6432686
  -r--r--r--  1 root  wheel   2744 Jan 11 04:13 ./#6432687
  -r--r--r--  1 root  wheel   2744 Jan 11 04:13 ./#6432688
  -r--r--r--  1 root  wheel   4587 Jan 11 04:13 ./#6432689
  -r--r--r--  1 root  wheel   4587 Jan 11 04:13 ./#6432690
  -r--r--r--  1 root  wheel   5877 Jan 11 04:13 ./#6432691
  -r--r--r--  1 root  wheel   5877 Jan 11 04:13 ./#6432692
  -r--r--r--  1 root  wheel   2423 Jan 11 04:13 ./#6432693
  -r--r--r--  1 root  wheel   2423 Jan 11 04:13 ./#6432694
  -rw-r--r--  1 root  wheel  36534 Jan 11 04:22 ./#6432776
  -rw-r--r--  1 root  wheel  36534 Jan 11 04:22 ./#6432777
  -rw-r--r--  1 root  wheel    348 Jan 11 04:22 ./#6432778
  -rw-r--r--  1 root  wheel    348 Jan 11 04:22 ./#6432779
  -rw-r--r--  1 root  wheel    357 Jan 11 04:22 ./#6432780
  -rw-r--r--  1 root  wheel    357 Jan 11 04:22 ./#6432781
  -rw-r--r--  1 root  wheel  15020 Jan 11 04:22 ./#6432782
  -rw-r--r--  1 root  wheel  15020 Jan 11 04:22 ./#6432783
  -rw-r--r--  1 root  wheel  12965 Jan 11 04:22 ./#6432784
  -rw-r--r--  1 root  wheel  12965 Jan 11 04:22 ./#6432785
  -r--r--r--  1 root  wheel    351 Jan 19 20:21 ./#6432788

mkdir lost+found
cd lost+found
(cd /mnt/lost+found ; tar cf - . ) | tar xf -
umount /dev/ad0s3d 
mount /dev/ad0s3d 
cd /mnt
rm -rf  lost+found
  rm: lost+found/#5393708: Directory not empty
  rm: lost+found: Directory not empty
cd /
umount /dev/ad0s3d
fsck -y /dev/ad0s3d
  ** /dev/ad0s3d
  ** Last Mounted on /mnt
  ** Phase 1 - Check Blocks and Sizes
  ** Phase 2 - Check Pathnames
  MISSING '.'  I=5393708  OWNER=root MODE=40755
  SIZE=512 MTIME=Mar  1 23:12 2005 
  DIR=/lost+found/#5393708
  
  UNEXPECTED SOFT UPDATE INCONSISTENCY
  
  FIX? yes
  
  ** Phase 3 - Check Connectivity
  ** Phase 4 - Check Reference Counts
  LINK COUNT DIR I=5393708  OWNER=root MODE=40755
  SIZE=512 MTIME=Mar  1 23:12 2005  COUNT 1 SHOULD BE 2
  LINK COUNT INCREASING
  UNEXPECTED SOFT UPDATE INCONSISTENCY
  
  ADJUST? yes
  
  ** Phase 5 - Check Cyl groups
  1436101 files, 20432208 used, 5476390 free (107758 frags, 671079 blocks, 0.4% fr
  agmentation)
  
  ***** FILE SYSTEM WAS MODIFIED *****

fsck -y /dev/ad0s3d	# no errors :-)

tunefs -p /dev/ad0s3d
  tunefs: ACLs: (-a)                                         disabled
  tunefs: MAC multilabel: (-l)                               disabled
  tunefs: soft updates: (-n)                                 enabled
  tunefs: maximum blocks per file in a cylinder group: (-e)  2048
  tunefs: average file size: (-f)                            16384
  tunefs: average number of files in a directory: (-s)       64
  tunefs: minimum percentage of free space: (-m)             8%
  tunefs: optimization preference: (-o)                      time
  tunefs: volume label: (-L)                                 

df -i /dev/ad0s3d
  Filesystem     Size    Used   Avail Capacity iused   ifree %iused  Mounted on
  /dev/ad0s3d     53G     42G    7.0G    86% 1436101 5276217   21%   /mnt

dumpfs /dev/ad0s3d > ~/dumpfs

How I (probably) created the damaged files system:
  Did several parallel NFS + AMD moves of trees with Many small
  files.  small being of size 2048 bytes, very many, as these were
  sliced from a few 4 Gig DVDs (while chasing a different suspect
  error on another system) The machine crashed a few time, but it
  wasnt warm at all on top of chassis, & mbmon reports cpu fan
  spinning OK, 
	Temp.= 29.0, 124.0, 25.0; Rot.= 2636, 2163,    0
	Vcore = 1.49, 2.56; Volt. = 3.34, 3.60, 11.61, -11.40, -4.58
  & I can see & feel air going in through chassis fan via extra big
  hole I cut in bottom front of chassis a couple of weeks back.
    A previous  Feb 7 mbmon had a lower voltage, dont know significance:
	Temp.= 34.0, 124.0, 25.0; Rot.= 2678, 2636,    0
	Vcore = 1.49, 2.56; Volt. = 3.34, 3.63, 11.61, -11.29, -3.66

If anyone want to create lots of small slice files:
	http://berklix.org/~jhs/src/bsd/jhs/bin/public/slice/

If there's any other info I should provide please say, Thanks.
-
Julian Stacey        Net & Sys Eng Consultant, Munich       http://berklix.com
 Mail in Ascii (Html=Spam).  Ihr Rauch = mein allergischer Kopfschmerz.


More information about the freebsd-fs mailing list