Re: ZFS replace a mirrored disk

Yes this partition layout created by bsdinstall.

----------------------------------

I continue with my tests:

Instead of `dd zero` all the disk, I boot on mfsBSD and clear metadata on both swap and zpool:

$> gmirror clear nvd0p2
$> zpool labelclear -f nvd0
$> gpart destroy -F nvd0

Then I boot on main OS and I run:

$> gpart backup nvd1 | gpart restore -F nvd0
$> gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 nvd0


$> gmirror status
       Name    Status  Components
mirror/swap  DEGRADED  nvd1p2 (ACTIVE)

So the gmirror status is as expected, and I have to forget / insert:

$> gmirror forget swap
$> gmirror insert swap /dev/nvd0p2

And the swap mirror is ready:

$> gmirror status
       Name    Status  Components
mirror/swap  COMPLETE  nvd1p2 (ACTIVE)
                       nvd0p2 (ACTIVE)

Now let's see the zpool:

$> zpool status
  pool: zroot
 state: DEGRADED
status: One or more devices could not be opened.  Sufficient replicas exist for
        the pool to continue functioning in a degraded state.
action: Attach the missing device and online it using 'zpool online'.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-2Q
config:

        NAME        STATE     READ WRITE CKSUM
        zroot       DEGRADED     0     0     0
          mirror-0  DEGRADED     0     0     0
            nvd0p3  UNAVAIL      0     0     0  cannot open
            nvd1p3  ONLINE       0     0     0

errors: No known data errors

I was expecting to asked to run "zpool replace" and not "zpool online", but it looks like it thinks it's a temporary error.

$> zpool online zroot nvd0p3

$> zpool status
  pool: zroot
 state: ONLINE
  scan: resilvered 8.93M in 00:00:00 with 0 errors on Thu May 12 13:08:12 2022
config:

        NAME        STATE     READ WRITE CKSUM
        zroot       ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            nvd0p3  ONLINE       0     0     0
            nvd1p3  ONLINE       0     0     0

I guess because I didn't `dd zero` resilvering fixes only the blocks that are changed, so not many MB.


------------------------------------

I also tried "dd if=/dev/zero of=/dev/nvd0 bs=1M" but when I boot on main OS it still show to use "zpool online" instead of "zpool replace" :

$> zpool status
  pool: zroot
 state: DEGRADED
status: One or more devices could not be opened.  Sufficient replicas exist for
        the pool to continue functioning in a degraded state.
action: Attach the missing device and online it using 'zpool online'.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-2Q
  scan: resilvered 9.14M in 00:00:00 with 0 errors on Thu May 12 14:31:40 2022
config:

        NAME                      STATE     READ WRITE CKSUM
        zroot                     DEGRADED     0     0     0
          mirror-0                DEGRADED     0     0     0
            11739263350767631641  UNAVAIL      0     0     0  was /dev/nvd0p3
            nvd1p3                ONLINE       0     0     0

errors: No known data errors