Bad blocks & 3ware RAID rebuild -- resolution

[Matt Staroscik]

I'm answering myself since (with the help of the list) I got my problem 
solved. I hope this summary helps a future searcher. Thanks to Jerry and 
everyone else who posted.

Summary of problem: My RAID mirror (3ware 7000-2) lost a drive, and it was 
failing to rebuild with a new replacement disk (under 4.10). I was not sure 
why the rebuild failed, due to a vague "disk error" message.

At this time I dumped the filesystems to a spare drive, but dump reported a 
read error on /usr. I fsck'd, but this did not eliminate the problem. 
Great, my degraded RAID's one "good" disk is having trouble!

It was pointed out to me that fsck would not fix bad blocks. I had to try 
something else to repair the disk. I located a corrupt file by tarring up 
/usr and waiting for trouble... A forgotten core dump showed a read error. 
I deleted it, and a subsequent dump of /usr did not report errors. 
Progress! However, the RAID would still not rebuild.

It turns out that an IDE drive will not remap a bad sector when READ... it 
must be WRITTEN. So, after I deleted the file that (presumably) sat on the 
bad sectors I filled up /usr with misc files. Afterwards, I checked the 
disk SMART error log and sure enough, it showed some new bad sectors had 
been remapped. (I checked disk 0 on the RAID with smartctl -a -d 3ware,0 
/dev/twed0)

I deleted the temporary files and initiated a RAID rebuild in the 3dm web 
interface. Before the sector repair, it would die after just a couple of 
minutes, but after the sector repair the rebuild completed successfully.

SMART still reports that the drive is overall healthy... I am not sure how 
many spare sectors are left but I have a working mirror again so I have 
bought myself some time.

Thanks again.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
matt at wrongcrowd.com * KF6IYW * http://wrongcrowd.com
"I am Matt Staroscik and I approved this message."