ZFS and reordering drives

[James R. Van Artsdalen]

Pawel Jakub Dawidek wrote:
> but there was a small
> bug, which I fixed yesterday, where guid wasn't checked if name matched.

After looking at the change and thinking about this I think there is
still a problem.

Consider a pool consisting of da1, da2 and da3.  Assume da2 fails hard
such that it is no longer seen by the system.

When the pool is imported da1 is attached by PATH since the GUID matches
when zpool.cache expects.

But when da2 is attached:

1. Attach by PATH & GUID fails even though there is a /dev/da2 (what was
da3) the GUID does not match.

2. Attach by GUID using GEOM search fails since da2 is no longer visible
and no disk has that GUID.

3. Attach by PATH falsely succeeds since the PATH exists even though the
GUID is wrong.

Furthermore, attempts to attach da3 fail since it was already attached
above.  The result, at least, might be a RAIDZ that should only DEGRADED
being FAULTED instead.

I don't see any way to fix this without callers specifying whether a
device is being added to a pool (don't check GUID) or merely attached as
part of a pool (always check GUID).