How to speed up slow zpool scrub?

[Brandon J. Wandersee]

Miroslav Lachman writes:

> krad wrote on 04/26/2016 16:02:
>> Erk, i would try to move your system off those data disks as you have
>> two pools competing for the disk spindles. This is never ideal. You can
>> by all means backup your os to those data pools but keep them on
>> separate physical mediums. A couple of small SSD would do the trick
>> nicely and could probably be added with no down time. You would probably
>> want to find a suitable window though to make sure the box reboots
>> nicely though.
>
> The system pool is really small - only 15GB and scrub is done relatively 
> fast. This machine cannot handle additional disks so I cannot move 
> system to other devices anyway. I tried system on USB flashdisk (read 
> only) in the past but it was slow and USB disk broke early.
>

I see no sign that the partitions are encrypted, so I can't think of a
reason that the system and data can't reside on the same pool. The
physical effort of the read arms on the disks isn't the only resource
to consider---each pool has its own ARC, for example, and if you have
compression enabled each pool will compete for CPU cycles. Having two
pools grabbing at such limited resources is going to noticeably hurt
performance.

All that said, running even a single relative large pool on a
low-powered dual-core CPU and 5G of RAM will noticeably limit *every*
ZFS operation. I would bet that while you might be able to shave some
time off the scrub, at best it's still going to take at least an entire
day on that hardware.

As a side note, I'm not so sure it's a good idea to use two different
redundancy methods on the same set of disks, though I can't say exactly
what the downsides would be (if any). It just seems odd.

-- 

::  Brandon J. Wandersee
::  brandon.wandersee at gmail.com
::  --------------------------------------------------
::  'The best design is as little design as possible.'
::  --- Dieter Rams ----------------------------------