Problem adding second mirror to zpool

[David J. Weller-Fahy]

I just realigned my zpool from ashift=9 to ashift=12, successfully 
transmitted the snapshot, and am booting from the new pool... however, I 
cannot get the last disk added to the pool.  Details follow (uname 
wrapped).

#v+
dave at nastie:~ $ uname -a
FreeBSD nastie.weller-fahy.com 10.1-RELEASE FreeBSD 10.1-RELEASE #0 r274551:
	Sun Nov 16 08:06:47 CST 2014
	root at nastie.weller-fahy.com:/usr/obj/usr/src/sys/GENERIC  amd64
#v-

This machine is a headless NAS, and has been updated since 9.?.

I had a ZFS pool which looked (approximately) like the following (prior 
to the realignment).  Note that the table is from memory, because I 
didn't keep a screenshot, but the device and partition names and numbers 
are correct.  The disks ada0 and ada2 are 1.5 TiB, and ada1 and ada3 are 
500 MiB.

#v+
NAME        STATE     READ WRITE CKSUM
nastiez     ONLINE       0     0     0
  mirror-0  ONLINE       0     0     0
    ada0p3  ONLINE       0     0     0
    ada2p3  ONLINE       0     0     0
  mirror-1  ONLINE       0     0     0
    ada1p1  ONLINE       0     0     0
    ada3p1  ONLINE       0     0     0
#v-

I detached ada2p3 and ada3p1 from the pool after scrubbing the pool, 
then created a new pool with ashift=12 with ada2p3 and ada3p1.  
Following that I had the following zpool status (just the table - these 
IDs are correct).

#v+
NAME                                          STATE     READ WRITE CKSUM
pool                                          ONLINE       0     0     0
  gptid/adc0e6d4-78a6-11e4-8646-0014222d98da  ONLINE       0     0     0
  gptid/c6ede2a4-78a6-11e4-8646-0014222d98da  ONLINE       0     0     0
#v-

Now, I transfered the snapshot from nastiez (the old zpool) to pool (the 
new zpool), verified the transfer, and destroyed the old zpool.  Next I 
duplicated the partition configuration that I used on ada2 and ada3 
on ada0 and ada1, and started to attach them to the pool.

First I did the following.

#v+
zpool attach pool \
	/dev/gptid/adc0e6d4-78a6-11e4-8646-0014222d98da \
	/dev/gptid/f0668af9-7aa2-11e4-8452-0014222d98da
#v-

Which made a mirror out of ada1p3 and ada0p3 (first and second, 
respectively) as shown.

#v+
NAME                                            STATE     READ WRITE CKSUM
pool                                            ONLINE       0     0     0
  mirror-0                                      ONLINE       0     0     0
    gptid/adc0e6d4-78a6-11e4-8646-0014222d98da  ONLINE       0     0     0
    gptid/f0668af9-7aa2-11e4-8452-0014222d98da  ONLINE       0     0     0
  gptid/c6ede2a4-78a6-11e4-8646-0014222d98da    ONLINE       0     0     0
#v-

Now I just need to mirror ada3p1 (gptid/c6ede...98da) with ada1p1, and 
I'll be done.  However (command line and error message wrapped)...

#v+
root at nastie:~ # zpool attach pool \
	/dev/gptid/c6ede2a4-78a6-11e4-8646-0014222d98da \
	/dev/gptid/fadbb43d-7aa2-11e4-8452-0014222d98da
cannot attach /dev/gptid/fadbb43d-7aa2-11e4-8452-0014222d98da to \
	/dev/gptid/c6ede2a4-78a6-11e4-8646-0014222d98da: no such device in pool
#v-

I've searched for about 2 days, and have not been able to find anything 
that either solves my problem or tells me why I can't add this device.

Can anyone tell me how to add the last drive (ada1) without losing the 
data on the drive?  Or is this a situation where I'll need to get a set 
of larger hard drives to get the data off this pool, and then 
re-organize after the data is safe?

Regards,
-- 
dave [ please don't CC me ]
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://lists.freebsd.org/pipermail/freebsd-questions/attachments/20141204/e5796c06/attachment.sig>