A seperate ZFS question

[Trond Endrestøl]

On Fri, 28 Apr 2017 14:14+0100, Paul Macdonald wrote:

> 
> I fat fingered adding a cache drive and was unable to remove the SSD cache
> drive (ada2)
> 
> I added a 2nd SSD (ada3) for safety as i didn;t want to risk the pool at all
> and exporting wasn;t feasible in timescale.
> 
> Can someone explain :
> 
>             what i've actually managed to set up here (!),
> 
>             the ramifications of forceably offlining/removing ada2
> 
>             and if i should worry when ada2 gets full?
> 
>                  capacity     operations    bandwidth
> pool          alloc   free   read  write   read  write
> ------------  -----  -----  -----  -----  -----  -----
> zrootnew       692G   460G     15    174   351K  2.40M
>   mirror       683G   237G     14    138   328K  1.43M
>     gpt/zfs0      -      -      6     50   155K  1.44M
>     ada1p3        -      -      7     50   176K  1.44M
>   ada2        9.17G   223G      0     36  22.9K   989K
> cache             -      -      -      -      -      -
>   ada3        39.0G   194G     23      5   168K   264K

First, you have gpt/zfs0 and ada1p3 in a mirrored configuration. Next, 
you have ada2 striped (RAID 0) to the first pair of mirrors. This is a 
very weird "RAID 1 + 0" combo. Guess what will happen when you lose 
ada2!

Maybe you should describe what you wanted to happen in the first 
place.

If you really wanted a three-way mirror, you have just learned the 
hard way about the difference between zpool add and zpool attach.

The latter is for adding additional members to existing mirrors, while 
the former is for adding raidz{1,2,3} groups to a pool, and the add 
command got you into this mess.

"zpool attach zrootnew ada1p3 ada2" is the right command if you wanted 
to create a three-way mirror in the first place.

I suggest you make good backups of this pool, destroy it, recreate it 
properly, and restore the missing data.

-- 
+-------------------------------+------------------------------------+
| Vennlig hilsen,               | Best regards,                      |
| Trond Endrestøl,              | Trond Endrestøl,                   |
| IT-ansvarlig,                 | System administrator,              |
| Fagskolen Innlandet,          | Gjøvik Technical College, Norway,  |
| tlf. mob.   952 62 567,       | Cellular...: +47 952 62 567,       |
| sentralbord 61 14 54 00.      | Switchboard: +47 61 14 54 00.      |
+-------------------------------+------------------------------------+