dump/restore corrupted filesystems

[CyberLeo Kitsana]

Roland Smith wrote:
>> Sorry if I wasn't clear. Most all of the data is readable and complete
>> if I mount the filesystem read-only. It just panics the box when mounted
>> read/write, and fsck can't fix the damage.
> 
> That might be worth filing a PR for, especially the panics. 
> 
> Exactly what is damaged?  Garbage in files? Wrong inode counts? I've had
> unclean filesystems because of panics, but nothing fsck_ffs couldn't
> fix.
> 
> You might want to check the hardware too. Use smartmontools in case of
> (S)ATA drives.

Smart says that the drives are fine, as does the manufacturer's disk
fitness tools. All the files that are readable contain correct data, but
the files that are corrupt are totally not readable, and cannot even be
removed manually:

--8<--
rsync: readlink
"/raid/Backup/Pizzabox/2007-02-23/cyberleo/secondlife/linux/SecondLife_i686_1_13_2_15/skins/xui/es"
failed: Bad file descriptor (9)
rsync: readlink
"/raid/Backup/Pizzabox/2007-02-23/cyberleo/secondlife/linux/SecondLife_i686_1_13_2_15/skins/xui/fr"
failed: Bad file descriptor (9)
--8<--

fsck_ufs dies after about 30 minutes of grinding with the following:

--8<--
** Phase 2 - Check Pathnames
DIRECTORY CORRUPTED  I=93409222  OWNER=1002 MODE=40755
SIZE=512 MTIME=Feb 10 00:49 2007
DIR=?

UNEXPECTED SOFT UPDATE INCONSISTENCY

SALVAGE? no

MISSING '.'  I=93409222  OWNER=1002 MODE=40755
SIZE=512 MTIME=Feb 10 00:49 2007
DIR=?

UNEXPECTED SOFT UPDATE INCONSISTENCY
CANNOT FIX, FIRST ENTRY IN DIRECTORY CONTAINS

UNEXPECTED SOFT UPDATE INCONSISTENCY
fsck_ufs: inoinfo: inumber -1170056596 out of range
--8<--

(full output is at
http://home.cyberleo.net/cyberleo/workspace/Zip/Bugs/fbsd-20070320-corr/saba-fsck-raid.txt
)

It's possible this might be a result of the odd interaction between
geom_raid5 and UFS, as discovered in January (
http://www.nabble.com/geom_raid5-livelock--p8304142.html ), but I can't
be sure.


I've already chalked this up to just an unfortunate occurrence, as the
circumstances that caused the corruption in the first place are likely
either long gone or so obscure as to be nearly impossible for me to root
out.

> Looking at /usr/src/sbin/dump/traverse.c, dump traverses the used
> inodes list and all directories. So if any of these is corrupt, your
> dump will be too. And if the contents of the inodes is corrupted, so
> will the dump.

Thanks for this insight. I'll avoid dump/restore and just use manual
copying for now.

--
Fuzzy love,
-CyberLeo
Technical Administrator
CyberLeo.Net Webhosting
http://www.CyberLeo.Net
<CyberLeo at CyberLeo.Net>

Furry Peace! - http://www.fur.com/peace/