How to speed up slow zpool scrub?

[Miroslav Lachman]

krad wrote on 04/26/2016 16:02:
> Erk, i would try to move your system off those data disks as you have
> two pools competing for the disk spindles. This is never ideal. You can
> by all means backup your os to those data pools but keep them on
> separate physical mediums. A couple of small SSD would do the trick
> nicely and could probably be added with no down time. You would probably
> want to find a suitable window though to make sure the box reboots
> nicely though.

The system pool is really small - only 15GB and scrub is done relatively 
fast. This machine cannot handle additional disks so I cannot move 
system to other devices anyway. I tried system on USB flashdisk (read 
only) in the past but it was slow and USB disk broke early.

> On 26 April 2016 at 14:35, Miroslav Lachman <000.fbsd at quip.cz
> <mailto:000.fbsd at quip.cz>> wrote:
>
>     InterNetX - Juergen Gotteswinter wrote on 04/26/2016 15:09:
>
>         to speed up the scrub itself you can try
>
>         sysctl -w vfs.zfs.scrub_delay = 4 (default, 0 means higher prio)
>
>
>     I will try it in the idle times
>
>         but be careful as this can cause a serious performance impact,
>         the value
>         can be changed on the fly
>
>         your pool is raidz, mirror ? dedup is hopefully disabled?
>
>
>     I forgot to mention it. Disks are partitioned to four partitions:
>
>     # gpart show -l ada0
>     =>        34  7814037101  ada0  GPT  (3.6T)
>                34           6        - free -  (3.0K)
>                40        1024     1  boot0  (512K)
>              1064    10485760     2  swap0  (5.0G)
>          10486824    31457280     3  disk0sys  (15G)
>          41944104  7769948160     4  disk0tank0  (3.6T)
>        7811892264     2144871        - free -  (1.0G)
>
>     diskXsys partitions are used for base system pool which is 4-way mirror
>
>     diskXtank0 partitions are used for data storage as RAIDZ
>
>     # zpool list
>     NAME    SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH
>     ALTROOT
>     sys    14.9G  11.0G  3.92G         -    79%    73%  1.00x  ONLINE  -
>     tank0  14.4T  10.8T  3.56T         -    19%    75%  1.00x  ONLINE  -
>
>
>     # zpool status -v
>        pool: sys
>       state: ONLINE
>        scan: scrub repaired 0 in 1h2m with 0 errors on Sun Apr 24
>     04:03:54 2016
>     config:
>
>              NAME              STATE     READ WRITE CKSUM
>              sys               ONLINE       0     0     0
>                mirror-0        ONLINE       0     0     0
>                  gpt/disk0sys  ONLINE       0     0     0
>                  gpt/disk1sys  ONLINE       0     0     0
>                  gpt/disk2sys  ONLINE       0     0     0
>                  gpt/disk3sys  ONLINE       0     0     0
>
>     errors: No known data errors
>
>        pool: tank0
>       state: ONLINE
>        scan: scrub in progress since Sun Apr 24 03:01:35 2016
>              7.63T scanned out of 10.6T at 36.7M/s, 23h32m to go
>              0 repaired, 71.98% done
>     config:
>
>              NAME                STATE     READ WRITE CKSUM
>              tank0               ONLINE       0     0     0
>                raidz1-0          ONLINE       0     0     0
>                  gpt/disk0tank0  ONLINE       0     0     0
>                  gpt/disk1tank0  ONLINE       0     0     0
>                  gpt/disk2tank0  ONLINE       0     0     0
>                  gpt/disk3tank0  ONLINE       0     0     0
>
>     errors: No known data errors
>
>
>     # zdb | grep ashift
>                  ashift: 12
>                  ashift: 12
>
>
>     Thank you for your informations.
>
>