How to speed up slow zpool scrub?

[Miroslav Lachman]

Andy Farkas wrote on 04/27/2016 06:39:
>> -----Original Message-----
>> From: PK1048 [mailto:paul at pk1048.com]
>> Sent: Wednesday, 27 April 2016 12:34 PM
>> To: Andy Farkas <andyf at andyit.com.au>
>> Cc: freebsd-fs at freebsd.org
>> Subject: Re: How to speed up slow zpool scrub?
>>
>> ...
>> Scrubs (and resilver) operations are essentially all random I/O. Those
>> drives are low end, low performance, desktop drives.
>
> Yes, the system is an old low end, low performance desktop. That was
> my point, that it took 25 hours to scrub 7.52T and not 4 days like the
> OP is saying.

Thank you for output of your zpool scrub. It is definitely faster than mine.

To: Paul pk1048
Mine scrub does not repair anything. Drives are OK (in SMART).
CPU is about 70%-90% idle during scrub + rsync backup and drives are 
about 60%-70% busy according to iostat:

root at kiwi ~/# iostat -x -w 10 ada0 ada1 ada2 ada3
                         extended device statistics
device     r/s   w/s    kr/s    kw/s qlen svc_t  %b
ada0      70.1  17.9  1747.0   802.4    0   7.0  24
ada1      70.1  17.9  1747.1   802.4    0   7.0  25
ada2      66.9  17.1  1686.4   791.3    4   6.5  23
ada3      66.9  16.9  1686.3   790.1    2   6.6  23
                         extended device statistics
device     r/s   w/s    kr/s    kw/s qlen svc_t  %b
ada0      93.6  13.0   576.4   244.0    0  21.6  70
ada1      98.6  12.9   587.2   246.4    2  20.5  71
ada2      87.9  15.3   566.0   242.4    3  20.5  67
ada3      84.9  14.5   549.2   237.2    3  20.4  66
                         extended device statistics
device     r/s   w/s    kr/s    kw/s qlen svc_t  %b
ada0      98.7  42.5  1924.7  2536.3    1  26.3  86
ada1      99.1  45.5  1931.5  2671.5    1  23.8  87
ada2      94.2  44.9  1840.7  2720.3    0  20.1  76
ada3      93.6  42.7  1807.9  2607.1    0  18.7  75
                         extended device statistics
device     r/s   w/s    kr/s    kw/s qlen svc_t  %b
ada0     108.2  28.2  1092.6  1316.6    2  17.3  68
ada1     101.6  26.3  1053.8  1183.4    3  15.5  67
ada2      98.6  26.0  1000.2  1126.2    2  12.2  57
ada3     104.0  24.0  1015.8  1080.6    3  14.1  60
                         extended device statistics
device     r/s   w/s    kr/s    kw/s qlen svc_t  %b
ada0     116.0  18.5   821.8   807.8    0  12.9  62
ada1     117.2  18.5   822.2   807.0    0  13.5  63
ada2     110.8  20.9   743.0   803.8    0  11.1  58
ada3     108.2  20.0   688.2   755.0    2  11.3  55
                         extended device statistics
device     r/s   w/s    kr/s    kw/s qlen svc_t  %b
ada0     121.8  16.6   602.1   526.9    3   9.2  52
ada1     122.2  16.5   606.9   528.5    4   9.8  54
ada2     117.0  14.6   601.7   524.9    2  11.3  60
ada3     120.6  13.5   610.1   491.3    0  11.4  61

I really don't know why it cannot go faster if nothing is loaded for 100%.

Miroslav Lachman