zpool remove not working for metadata special devices

John Delisle jdelisle at gmail.com
Mon Dec 28 00:55:18 UTC 2020


I have a pool of mirrors, and added a mirrored special device.  Although
documentation suggests it should be removable, I cannot get zpool to do so.

All top-level vdevs are mirrors, and all have the same sector size and
ashift.  No raidz.

## Current 12.2 p2
root at jmdtest:/ # freebsd-version
12.2-RELEASE-p2

## Disk info
root at jmdtest:/ # diskinfo /dev/da[2-9] /dev/da[0-9][0-9]
/dev/da2        512     274877906944    536870912       4096    0
33418   255     63
/dev/da3        512     274877906944    536870912       4096    0
33418   255     63
/dev/da4        512     274877906944    536870912       4096    0
33418   255     63
/dev/da5        512     274877906944    536870912       4096    0
33418   255     63
/dev/da6        512     274877906944    536870912       4096    0
33418   255     63
/dev/da7        512     274877906944    536870912       4096    0
33418   255     63
/dev/da8        512     274877906944    536870912       4096    0
33418   255     63
/dev/da9        512     274877906944    536870912       4096    0
33418   255     63
/dev/da10       512     274877906944    536870912       4096    0
33418   255     63
/dev/da11       512     274877906944    536870912       4096    0
33418   255     63
/dev/da12       512     68719476736     134217728       4096    0
8354    255     63
/dev/da13       512     68719476736     134217728       4096    0
8354    255     63


## Create the pool:
zpool create nebula mirror da2 da3 mirror da4 da5 mirror da6 da7 mirror da8
da9 mirror da10 da11

## Add the special mirror
zpool add nebula special mirror da12 da13

## zpool status
root at jmdtest:/ # zpool list -v nebula
NAME         SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP
 HEALTH  ALTROOT
nebula      1.30T  2.57G  1.30T        -         -     0%     0%  1.00x
 ONLINE  -
  mirror     254G   492M   254G        -         -     0%  0.18%
    da2         -      -      -        -         -      -      -
    da3         -      -      -        -         -      -      -
  mirror     254G   501M   254G        -         -     0%  0.19%
    da4         -      -      -        -         -      -      -
    da5         -      -      -        -         -      -      -
  mirror     254G   596M   253G        -         -     0%  0.22%
    da6         -      -      -        -         -      -      -
    da7         -      -      -        -         -      -      -
  mirror     254G   481M   254G        -         -     0%  0.18%
    da8         -      -      -        -         -      -      -
    da9         -      -      -        -         -      -      -
  mirror     254G   561M   253G        -         -     0%  0.21%
    da10        -      -      -        -         -      -      -
    da11        -      -      -        -         -      -      -
special         -      -      -         -      -      -
  mirror    63.5G  2.62M  63.5G        -         -     0%  0.00%
    da12        -      -      -        -         -      -      -
    da13        -      -      -        -         -      -      -
root at jmdtest:/ #

## Remove the special mirror
root at jmdtest:/ # zpool remove nebula mirror-5
cannot remove mirror-5: invalid config; all top-level vdevs must have the
same sector size and not be raidz.

## All mirrors, and all the same ashift:
root at jmdtest:/ # zdb -C | grep -e child -e ashift
    vdev_children: 6
        children[0]:
            ashift: 12
            children[0]:
            children[1]:
        children[1]:
            ashift: 12
            children[0]:
            children[1]:
        children[2]:
            ashift: 12
            children[0]:
            children[1]:
        children[3]:
            ashift: 12
            children[0]:
            children[1]:
        children[4]:
            ashift: 12
            children[0]:
            children[1]:
        children[5]:
            ashift: 12
            children[0]:
            children[1]:
root at jmdtest:/ #

What am I doing wrong?


More information about the freebsd-fs mailing list