ZFS and disk naming change (ex. da0->da4)

[Scott Long]

Pawel Jakub Dawidek wrote:
> On Wed, Oct 24, 2007 at 04:48:38PM +0200, Attila Nagy wrote:
>> Hello,
>>
>> I have an experimental (but that does not mean, I wouldn't like to get 
>> my data back :) zpool, which was created with something like this:
>> zpool create people raidz2 /dev/da0 /dev/da3 /dev/da4, etc
>>
>> The problem is those device names have been changed during the next 
>> reboot (the cause of this is irrelevant, but mainly because some of them 
>> were not attached at the original boot, just later, so at the next 
>> reboot the disks came up in a different order), so now I have:
>> zpool status
>>  pool: people
>> state: UNAVAIL
>> status: One or more devices could not be opened.  There are insufficient
>>        replicas for the pool to continue functioning.
>> action: Attach the missing device and online it using 'zpool online'.
>>   see: http://www.sun.com/msg/ZFS-8000-D3
>> scrub: none requested
>> config:
>>
>>        NAME        STATE     READ WRITE CKSUM
>>        people      UNAVAIL      0     0     0  insufficient replicas
>>          raidz2    UNAVAIL      0     0     0  insufficient replicas
>>            da0     UNAVAIL      0     0     0  cannot open
>>            da3     ONLINE       0     0     0
>>            da4     FAULTED      0     0     0  corrupted data
>>            da5     FAULTED      0     0     0  corrupted data
>>            da6     FAULTED      0     0     0  corrupted data
>>            da7     FAULTED      0     0     0  corrupted data
>>            da8     FAULTED      0     0     0  corrupted data
>>            da9     FAULTED      0     0     0  corrupted data
>>
>> (it seems da3 is still da3 :)
>>
>> My question is: what now? Is it possible to regain the pool, or is it 
>> totally busted now? I am not sure that I can figure out which device is 
>> which now...
>>
>> I've only played with ZFS on Solaris with FC targets, and there I've 
>> never faced this problem, because of the static naming.
> 
> ZFS caches components names in /boot/zfs/zpool.cache. You may remove
> this file and import the pool again.
> 
> ZFS can handle name changes, but currently only with ATA disks. It can
> find disk using it's ID. I've a patch for SCSI disks, but it's probably
> not entirely correct:
> 
> 	http://people.freebsd.org/~pjd/patches/scsi_da_ident.patch
> 
> We need some SCSI guru to implement it properly.
> 
>> ps: I guess next time I will use glabel -I love that- to provide base 
>> devices...
> 
> Yes, glabel seems like a good work-around for now. In some cases we will
> never be able to provide disk's ID, eg. for USB pen-drives, etc.
> 

Serial numbers are not reliably unique in SCSI.  Identifying SCSI disks
in a unique way has been a problem for decades since such uniqueness was
never mandated in any specs and the vendors never agreed on if or how to
do it in a uniform way.  The best you can do is to look at the data
from VPD pages 0x81 and 0x83 and hueristically guess if what you're told
is usable.

SAS is the first SCSI-like technology that has mandated unique device
ID's; I believe that FC only guarantees unique port ID's, but for a
multi-port FC device, it's still a guess as to whether you'll be able to
determine device uniqueness.  However, since multi-pathing is such a
desired feature with FC, it's a good bet these days that you'll be able
to extract a unique device ID from each device.

If you'd like some example code on how to do VPD queries from userland
or even from the kernel, I'd be happy to show you.

Scott