Re: ZFS - reboot during resilver doesn't work

On 9/8/25 11:26, Frank Leonhardt wrote:
> On 08/09/2025 05:08, David Christensen wrote:
>> The few times I have replaced and resilvered a HDD, the process took 
>> about 4 hours per TB of data.  The disks were 3 TB SATA Seagate 
>> Barracuda or Constellation ES.2.
>>
>>
>> If you are wondering if the existing drive and/or the replacement 
>> drive are having issues, perhaps running `smartctl -x ...` could 
>> confirm or refute.  And, look at /var/log/messages for errors.
> 
> Yes, that would be about right on the speed.


For long operations, I often open a pair of terminals for each ZFS host. 
  In one:

# top -S -s 60


In the other:

# zpool iostat -v mypool 60


> For everyone's amusement, it re-silvered 900Gb of data in about nine 
> hours, which could be worse. I'm convinced, as Doron concurred, it was 
> down to the replacement being SMR. These days I test new drives 
> thoroughly before heading to a data centre with one, and interestingly 
> it was pretty good on sequential read/write - 250MB/s sustained. No 
> flaws. Re-silvering it managed about 20Mb/s according to  iostat. 
> According to zfs status it was doing about 150B/s (that's bytes).


I agree that the `zpool status` statistics can be confusing.  AIUI some 
of the statistics are reported from the start of the replace/ scrub/ 
etc. operation while others are soft real-time (e.g. the past few seconds).


> There are clearly a few bugs here - both in what zfs status reports and 
> in shutting down the re-silver to reboot. I'm investigating offline. SMR 
> drives are bad news.


AIUI SMR drive random write performance is a bottleneck.  I assume there 
are good use-cases for SMR drives, but I am not using them.


I added a special vdev mirror of two SSD's to my pools during the last 
rebuild.  Subjective operational responsiveness improved noticeably. 
Scrubs take less time.  Replication is better able to fill the network 
connection.  I would expect a special vdev to reduce drive replacement time.


> Incidentally, if you do a zpool replace with a -s sequential option it 
> should go a lot quicker as it's normally doing the equivalent of a 
> scrub, rebuilding the mirror with a load of random writes. This is an 
> enhancement from OpenZFS and the reason for it is now very apparent.


I am curious to try sequential reconstruction (the next time I have the 
need).


> I shall be writing it up on my blog as usual. Now, do I dare reboot it?
> 
> Regards, Frank.
>
> <snip>


AIUI both your OS and data are on a ZFS mirror of two large HDD's (?). 
I find disaster recovery is easier when the OS is on a small SSD.  And, 
mirroring the OS device makes sense in many scenarios; including remote 
location.


David