danging dbufs in ZFS v6 , FreeBSD 7.1

[Bryant Eadon]

A problem I'm hoping you can solve !

Running on a 64bit platform with 5, 500GB HDDs in a basic raidz configuration 
classically named 'tank', I began copying a file.  During the copy, I lost a 
disk.  Since these are all hot swappable SATA drives, I pulled the one I 
*thought* had died and swapped in a good drive, which powered up and I attempted 
a 'replace'.  The copy was still proceeding... (you see where this is going...)

This wasn't the broken drive I pulled, which I quickly found after the replace 
attempt !  In an effort to put the good drive back into the array, I reconnected 
and rebooted the machine citing possible 'drive disappearance' problems with the 
stunt I just pulled.

Nothing doing.  The kernel hung at :

"panic :  dangling dbufs.
dn = 0xffffff000a49f338
dbuf = 0xffffff000a4a01e0 "

Leading me to believe the array is dead. :-/

I am happy to lose the data that was copied at the time of failure if it's 
possible to recover the rest of the array.

I suppose that the rest of the data remains intact.   Is there a way to rid 
myself of the dangling buffers to get back to a usable state ? (some dd magic ?)

Could I recover by using a newer version of ZFS for FreeBSD ? (v13 instead of v v6)

Quickly it moved from :
         NAME        STATE     READ WRITE CKSUM
         tank        DEGRADED     0     0     0
           raidz1    DEGRADED     0     0     0
             da0     ONLINE       0     0     0
             da1     UNAVAIL      0   887     0  cannot open
             da2     ONLINE       0     0     0
             ad5     ONLINE       0     0     0
             ad6     ONLINE       0     0     0

to :

         NAME        STATE     READ WRITE CKSUM
         tank        UNAVAIL      0     0     0  insufficient replicas
           raidz1    UNAVAIL      0     0     0  insufficient replicas
             da0     UNAVAIL      0     0     0  cannot open
             da1     UNAVAIL      0     0     0  cannot open
             da2     ONLINE       0     0     0
             ad5     ONLINE       0     0     0
             ad6     ONLINE       0     0     0