resilver slowdown....

[Michelle Sullivan]

Michelle Sullivan wrote:
> After retuning for priority to the resilver, I see massive (very good) 
> performance.... then at some random time later (or so it seemed) the 
> performance would drop off very quickly always at least to half the 
> performance, sometimes as much as a quarter of the speed...
>
>
> At first I though it was memory based, then observing system with 19G 
> (of 32G) free figured it must be based on the stuff on the disk got 
> some 16T of very small files, and another 16T of video files so 
> figured it was probably going from large files to small files etc...  
> Nope, after a reboot and resetting the tuning params to the same each 
> time after the reboot I'd be seeing an immediate and sustained 
> performance...  Watching it more it seems this is directly related:
>
> [root at colossus ~]# sysctl -a | grep \\.arc_
> vfs.zfs.arc_meta_limit: 4294967296
> vfs.zfs.arc_meta_used:  5529107352
> vfs.zfs.arc_min:  8589934592
> vfs.zfs.arc_max: 17179869184
Yup think it is related to the meta_limit and meta_used...

Resilvering at 203M/s (according to zpool status)
[root at colossus ~]# sysctl -a | grep \\.arc_
vfs.zfs.arc_meta_limit: 4294967296
vfs.zfs.arc_meta_used: 2187549528
vfs.zfs.arc_min: 8589934592
vfs.zfs.arc_max: 17179869184

Noticed it had slowed again.. around 64M/s ...
[root at colossus ~]# sysctl -a | grep \\.arc_
vfs.zfs.arc_meta_limit: 4294967296
vfs.zfs.arc_meta_used: 4295111952
vfs.zfs.arc_min: 8589934592
vfs.zfs.arc_max: 17179869184

Rebooted and back to >120M/s (can't get a reading atm - too soon after 
the reboot)

-- 
Michelle Sullivan
http://www.mhix.org/