N-way mirror read speedup in zfsonlinux

[Alexander Motin]

On 04.08.2013 19:37, Alexander Motin wrote:
> Hi.
>
> I like the idea of load-aware balancing. I went that way in both gmirror
> and graid and seen substantial performance improvements. But pure
> load-based balance may be ineffective in case of spinning HDDs and
> sequential I/Os. Doing that way will mitigate effect of the device
> read-ahead. If sequential read operations will be handled on triple
> mirror as ABCABC, we may get situation when all three devices are busy,
> but giving only read speed of one. vdev_mirror_shift used in the
> original balance code has its reason -- it will make all I/Os to the
> consequential offsets (within 2MB) go to the same disk. You may find
> G_RAID_SUBDISK_TRACK_SIZE constant in graid and TRACK_SIZE in gmirror
> doing exactly the same.
>
> I am not have doubt in effectiveness of proposed time-based magic. If

Sorry, I mean I DO have doubt ...

> there is only one request issued at a time, this algorithm indeed may
> give benefit by more effective using of the device read-ahead. But if we
> have more then one request active, the result will become purely random
> since first request sent will affect later ones and (d % lowest_nr) will
> give precedence to completely random device.
>
> Also I would like to note that getting precise time still can be
> expensive operation on some hardware to do it on every I/O. I think in
> many cases there will be only one device with lowest load, and reading
> of time could be skipped.
>
> On 04.08.2013 12:25, Martin Matuska wrote:
>> Attached is a FreeBSD version of this patch for testing and comments,
>> including sysctl tunable:
>> http://people.freebsd.org/~mm/patches/zfs/vdev_mirror.c.patch
>>
>> On 2013-07-12 11:21, Martin Matuška wrote:
>>> zfsonlinux has implemented a change in the N-way mirror device selection
>>> algorithm by selecting the device with the least pending I/O instead of
>>> random selection. They measured an increased read bandwidth increase
>>> up to
>>> 50% and IOPS increase up to 10%.
>>>
>>> this might be useful for common ZFS code and we might consider porting
>>> this
>>> to illumos and FreeBSD:
>>> https://github.com/zfsonlinux/zfs/issues/1461
>>> https://github.com/zfsonlinux/zfs/commit/556011dbec2d10579819078559a77630fc559112
>>>
>


-- 
Alexander Motin