ZFS stalled after some mirror disks were lost

[Ben RUBSON]

> On 03 Oct 2017, at 09:25, Steven Hartland <steven at multiplay.co.uk> wrote:
> 
> On 03/10/2017 07:12, Andriy Gapon wrote:
>> On 02/10/2017 21:12, Ben RUBSON wrote:
>> 
>>> A sustained read throughput of 180 MB/s, 45 MB/s on each iscsi disk
>>> according to "zpool iostat", nothing on local disks (strange but I
>>> noticed that IOs always prefer iscsi disks to local disks).
>>> 
>> Are your local disks SSD or HDD?
>> Could it be that iSCSI disks appear to be faster than the local disks to the
>> smart ZFS mirror code?
>> 
>> Steve, what do you think?
>> 
> Yes that quite possible, the mirror balancing uses the queue depth + rotating bias to determine the load of the disk so if your iSCSI host is processing well and / or is reporting non-rotating vs rotating for the local disks it could well be the mirror is preferring reads from the the less loaded iSCSI devices.

Note that local & iscsi disks are _exactly_ the same (same model number, same SAS adapter...).
So iSCSI ones should be a little bit slower due to network latency (even if it's very low in my case).
Once production back, after having analysed the main issue of this thread, I should then
try to find whether or not iSCSI disks are seen as rotating disks.

Thanks for the hint !

Ben