Ways to "pause" ZFS resilver?

[Warner Losh]

On Sun, Mar 8, 2020 at 12:35 PM Peter Eriksson <pen at lysator.liu.se> wrote:

> I’m looking for ideas on how to pause a running ZFS resilver on a FreeBSD
> 11.3-RELEASE-p6 system.
>
> The reason is we have a system where a running such causes severe NFS
> “hiccups” for our users (like 5-20s delays more or less often) and thus I’d
> like to figure out some way to “pause” it during office hours until either
> we’ve found and fixed the problem or the resilver is done (1D15H to go)...
>
> Since there isn’t any “zfs” command to pause a running resilver I’m
> pondering alternative more “creative” ways.
>
> /usr/src/cddl/contrib/opensolaris/uts/common/fs/zfs:
>
> >       if (zio_flags & ZIO_FLAG_RESILVER)
> >                scan_delay = zfs_resilver_delay;
> >        else {
> >                ASSERT(zio_flags & ZIO_FLAG_SCRUB);
> >                scan_delay = zfs_scrub_delay;
> >        }
> >
> >        if (scan_delay && (ddi_get_lbolt64() - spa->spa_last_io <=
> zfs_scan_idle))
> >                delay(MAX((int)scan_delay, 0));
>
> Settings vfs.zfs.scan_idle to something high and then
> vfs.zfs.resilver_delay to 10*60*60*kern.hz (10 hours) and hoping the “if"
> statement will trigger? But that assumes nothing can/will interrupt
> delay(). Hmmm...
>
> Any other suggestions?
>
> (I don’t want to abort the resilver).
>

If you are dealing with SSDs, you might look to see if BIO_DELETE (trim)
traffic is causing delays. If so, you can temporarily disable TRIM on the
disk being resilvered. In the resilver case, trim doesn't help much anyway
since you're rewriting the entire drive. If not, then I'm not sure what
else to recommend...

Warner