How to speed up slow zpool scrub?

[Miroslav Lachman]

InterNetX - Juergen Gotteswinter wrote on 04/26/2016 15:09:
> to speed up the scrub itself you can try
>
> sysctl -w vfs.zfs.scrub_delay = 4 (default, 0 means higher prio)

I will try it in the idle times

> but be careful as this can cause a serious performance impact, the value
> can be changed on the fly
>
> your pool is raidz, mirror ? dedup is hopefully disabled?

I forgot to mention it. Disks are partitioned to four partitions:

# gpart show -l ada0
=>        34  7814037101  ada0  GPT  (3.6T)
           34           6        - free -  (3.0K)
           40        1024     1  boot0  (512K)
         1064    10485760     2  swap0  (5.0G)
     10486824    31457280     3  disk0sys  (15G)
     41944104  7769948160     4  disk0tank0  (3.6T)
   7811892264     2144871        - free -  (1.0G)

diskXsys partitions are used for base system pool which is 4-way mirror

diskXtank0 partitions are used for data storage as RAIDZ

# zpool list
NAME    SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
sys    14.9G  11.0G  3.92G         -    79%    73%  1.00x  ONLINE  -
tank0  14.4T  10.8T  3.56T         -    19%    75%  1.00x  ONLINE  -


# zpool status -v
   pool: sys
  state: ONLINE
   scan: scrub repaired 0 in 1h2m with 0 errors on Sun Apr 24 04:03:54 2016
config:

         NAME              STATE     READ WRITE CKSUM
         sys               ONLINE       0     0     0
           mirror-0        ONLINE       0     0     0
             gpt/disk0sys  ONLINE       0     0     0
             gpt/disk1sys  ONLINE       0     0     0
             gpt/disk2sys  ONLINE       0     0     0
             gpt/disk3sys  ONLINE       0     0     0

errors: No known data errors

   pool: tank0
  state: ONLINE
   scan: scrub in progress since Sun Apr 24 03:01:35 2016
         7.63T scanned out of 10.6T at 36.7M/s, 23h32m to go
         0 repaired, 71.98% done
config:

         NAME                STATE     READ WRITE CKSUM
         tank0               ONLINE       0     0     0
           raidz1-0          ONLINE       0     0     0
             gpt/disk0tank0  ONLINE       0     0     0
             gpt/disk1tank0  ONLINE       0     0     0
             gpt/disk2tank0  ONLINE       0     0     0
             gpt/disk3tank0  ONLINE       0     0     0

errors: No known data errors


# zdb | grep ashift
             ashift: 12
             ashift: 12


Thank you for your informations.


> Am 4/26/2016 um 2:44 PM schrieb Miroslav Lachman:
>> Hi,
>>
>> is there any way to make zpool scrub faster?
>> We have one older machine with CPU Pentium(R) Dual E2160 @1.80GHz, 5GB
>> of RAM and 4x 4TB HDDs. It is just a storage for backups for about 20
>> machines.
>> Scrub is scheduled from periodic each 30 days but it takes about 4 days
>> to complete and everything during scrub is slow. Backups takes 8 hours
>> instead of 5 (made by rsync), deleting of old files is even more slower.
>>
>> The backups are made every night from the midnight to morning, the
>> machine is idle for the rest of the day.
>>
>> Is there any tuning to make scrub faster in this idle time?
>> Or is it better to do it other way - slower scrub with even lower
>> priority taking for about one week but not affecting time of normal
>> operations? (is it dangerous to have scrub running this long or reboot
>> machine during the scrub?)
>>
>> I have a performance graphs of this machine and CPU is about 70% idle
>> during scrub, but hard drives are busy 75% (according to iostat)
>>
>> FreeBSD 10.3-RELEASE amd64 GENERIC
>>
>> Miroslav Lachman
>> _______________________________________________
>> freebsd-fs at freebsd.org mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs
>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"
>