svn commit: r256956 - in head/sys: cam/ata cam/scsi cddl/contrib/opensolaris/uts/common/fs/zfs cddl/contrib/opensolaris/uts/common/fs/zfs/sys geom sys

[Slawa Olhovchenkov]

On Wed, Oct 23, 2013 at 09:54:59AM +0000, Steven Hartland wrote:

> Author: smh
> Date: Wed Oct 23 09:54:58 2013
> New Revision: 256956
> URL: http://svnweb.freebsd.org/changeset/base/256956
> 
> Log:
>   Improve ZFS N-way mirror read performance by using load and locality
>   information.
>   
>   The existing algorithm selects a preferred leaf vdev based on offset of the zio
>   request modulo the number of members in the mirror. It assumes the devices are
>   of equal performance and that spreading the requests randomly over both drives
>   will be sufficient to saturate them. In practice this results in the leaf vdevs
>   being under utilized.
>   
>   The new algorithm takes into the following additional factors:
>   * Load of the vdevs (number outstanding I/O requests)
>   * The locality of last queued I/O vs the new I/O request.
>   
>   Within the locality calculation additional knowledge about the underlying vdev
>   is considered such as; is the device backing the vdev a rotating media device.
>   
>   This results in performance increases across the board as well as significant
>   increases for predominantly streaming loads and for configurations which don't
>   have evenly performing devices.
>   
>   The following are results from a setup with 3 Way Mirror with 2 x HD's and
>   1 x SSD from a basic test running multiple parrallel dd's.
>   
>   With pre-fetch disabled (vfs.zfs.prefetch_disable=1):
>   
>   == Stripe Balanced (default) ==
>   Read 15360MB using bs: 1048576, readers: 3, took 161 seconds @ 95 MB/s
>   == Load Balanced (zfslinux) ==
>   Read 15360MB using bs: 1048576, readers: 3, took 297 seconds @ 51 MB/s
>   == Load Balanced (locality freebsd) ==
>   Read 15360MB using bs: 1048576, readers: 3, took 54 seconds @ 284 MB/s
>   
>   With pre-fetch enabled (vfs.zfs.prefetch_disable=0):
>   
>   == Stripe Balanced (default) ==
>   Read 15360MB using bs: 1048576, readers: 3, took 91 seconds @ 168 MB/s
>   == Load Balanced (zfslinux) ==
>   Read 15360MB using bs: 1048576, readers: 3, took 108 seconds @ 142 MB/s
>   == Load Balanced (locality freebsd) ==
>   Read 15360MB using bs: 1048576, readers: 3, took 48 seconds @ 320 MB/s
>   
>   In addition to the performance changes the code was also restructured, with
>   the help of Justin Gibbs, to provide a more logical flow which also ensures
>   vdevs loads are only calculated from the set of valid candidates.
>   
>   The following additional sysctls where added to allow the administrator
>   to tune the behaviour of the load algorithm:
>   * vfs.zfs.vdev.mirror.rotating_inc
>   * vfs.zfs.vdev.mirror.rotating_seek_inc
>   * vfs.zfs.vdev.mirror.rotating_seek_offset
>   * vfs.zfs.vdev.mirror.non_rotating_inc
>   * vfs.zfs.vdev.mirror.non_rotating_seek_inc
>   
>   These changes where based on work started by the zfsonlinux developers:
>   https://github.com/zfsonlinux/zfs/pull/1487
>   
>   Reviewed by:	gibbs, mav, will
>   MFC after:	2 weeks
>   Sponsored by:	Multiplay

No MFC yet.