zfs drive keeps failing between export and import

[Wes Morgan]

On Sat, 24 Jan 2009, David Ehrmann wrote:

> On Thu, Jan 22, 2009 at 8:07 PM, Wes Morgan <morganw at chemikals.org> wrote:
>> On Thu, 22 Jan 2009, David Ehrmann wrote:
>>
>>> On Fri, Jan 16, 2009 at 3:21 PM, David Ehrmann <ehrmann at gmail.com> wrote:
>>>
>>> In the /dev/ad8.eli that zfs doesn't recognize, I found a 16 byte
>>> string that was repeated a lot, but it was also repeated in another
>>> place: the good /dev/ad10.eli (though the offsets were different).
>>> The other weird thing: the good and bad /dev/ad8.eli look a lot alike:
>>> one 16 byte string, then another that gets repeated, then another 16
>>> byte string randomly shows up at 0x200.
>>
>> The "xxx.eli" devices are the decrypted versions, aren't they? ZFS vdev
>> labels and uberblocks occupy the first 512k of the device, and consist of
>> virtually identical data, differing only by the GUID that the label claims
>> to be and a sha256 checksum... So, decrypted, they should all be very, very
>> similar. You could actually use the label from any device in a pool to
>> reconstruct the label for any other device.
>
> Let me clarify one thing: when zfs has problems reading the device,
> the data resemble the data when it's read fine, but by resemble, I
> don't mean values as much as structure.  The values are all wrong, but
> if you overlaid hexdumps, both share repeated patterns.  That makes me
> think it's an encryption problem, but I haven't been able to reproduce
> it with other configurations.

You might try creating the pool, saving the first 512k of each block 
device to a file, then export the pool and repeat, then import (or try 
to). Run zdb on each file and compare the output. From creation to export 
to import they should only differ by the "state" in the top level of the 
label nvlist. If the entire label is corrupted, then likely it's a crypto 
problem.

Although, it really sounds like you've been able to eliminate zfs as a 
culprit.