hard disk failure - now what?

[Kelly Martin]

I just experienced a hard drive failure on one of my FreeBSD 7.2
production servers with no backup! I am so mad at myself for not
backing up!! Now it's a salvage operation. Here are the type of errors
I was getting on the console, over-and-over:

ad4: TIMEOUT - WRITE_DMA48 retrying (0 retries left) LBA=441633503
ad4: WARNING - SETFEATURES ENABLE RCACHE taskqueue timeout -
completing request directly
ad4: WARNING - SETFEATURES ENABLE WCACHE taskqueue timeout -
completing request directly
ad4: WARNING - SET_MULTI taskqueue timeout - completing request directly
ad4: FAILURE - WRITE_DMA48 timed out LBA=441633375
g_vgs_done():ad4s1f[WRITE(offset=216338284544, length=16384)]error = 5

I could still login to the machine (after an eternity) but got lots of
read/write errors along the way.  The offset shown in the errors kept
changing, so I thought it was a hardware eSATA controller issue
instead of a bad sector on the drive -  I replaced the motherboard,
but the problem persisted. So I bought a new hard drive and have
re-installed FreeBSD 7.2 on it. I'd like to plug in the old hard drive
today, mount it and salvage as much as I can... especially the
database files, config files, etc.

My question: what kind of checks and/or repair tools should I run on
the damaged drive after it's mounted? Or should I mount it as
read-only and start backing it up? I am hoping most of my data is
still there, but also don't want to damage it further. I desperately
need to salvage the data, what do the kind people on this list
recommend?

thanks,
kelly