zfs detach/replace

[Michael DeMan]

Right, I botched it and did a 'replace' on a drive that is in an raidz2 pool.  replace is for replacing a mirrored drive.

This happened to us a couple years ago during early testing, we noticed it, and at that time were unable to get the machine out of its funky state without destroying and rebuilding the pool.  Basically drive #5 in the raidz2 pool thinks that it is  supposed to be working on a replacement for a pair of mirrored physical drives.

We have shutdown and started the machine without the drive installed, done all sorts of things, etc.  We can not even offline it because:

<SNIP># zpool offline zp1rz2 label/ada5LABEL
cannot offline label/ada5LABEL: no valid replicas


On Dec 31, 2011, at 4:08 PM, Joshua Boyd wrote:

> On Sat, Dec 31, 2011 at 1:58 AM, Michael DeMan <freebsd at deman.com> wrote:
> Hi All,
> 
> The origination of the problem is entirely my fault on  FreeBSD 8.1 RELEASE #0.  We had old notes that attempting a 'replace' (which is appropriate for a mirror) leaves ZFS in a funky state on BSD.  I inadvertently did just that on a drive swap on a raidz2 pool.  My old notes show the only recovery that we knew of at the time was to rsync or zfs-send the pool elsewhere, destroy the local and rebuild from scratch.
> 
> I've never had a problem before, and have replaced about 5 drives in my striped raidz ...
> 
> Usually I'll execute a zpool offline, camcontrol stop, remove the drive, then zpool replace.
> 
> I am running 8-STABLE though, and not -RELEASE.
> 
> -- 
> Joshua Boyd
> 
> E-mail: boydjd at jbip.net
> http://www.jbip.net