svn commit: r256956 - in head/sys: cam/ata cam/scsi cddl/contrib/opensolaris/uts/common/fs/zfs cddl/contrib/opensolaris/uts/common/fs/zfs/sys geom sys

[Steven Hartland]

----- Original Message ----- 
From: "Slawa Olhovchenkov" <slw at zxy.spb.ru>
To: "Steven Hartland" <smh at FreeBSD.org>
Cc: <src-committers at freebsd.org>; <svn-src-all at freebsd.org>; <svn-src-head at freebsd.org>
Sent: Sunday, September 07, 2014 5:01 PM
Subject: Re: svn commit: r256956 - in head/sys: cam/ata cam/scsi cddl/contrib/opensolaris/uts/common/fs/zfs 
cddl/contrib/opensolaris/uts/common/fs/zfs/sys geom sys


> On Wed, Oct 23, 2013 at 09:54:59AM +0000, Steven Hartland wrote:
>
>> Author: smh
>> Date: Wed Oct 23 09:54:58 2013
>> New Revision: 256956
>> URL: http://svnweb.freebsd.org/changeset/base/256956
>>
>> Log:
>>   Improve ZFS N-way mirror read performance by using load and locality
>>   information.
>>
>>   The existing algorithm selects a preferred leaf vdev based on offset of the zio
>>   request modulo the number of members in the mirror. It assumes the devices are
>>   of equal performance and that spreading the requests randomly over both drives
>>   will be sufficient to saturate them. In practice this results in the leaf vdevs
>>   being under utilized.
>>
>>   The new algorithm takes into the following additional factors:
>>   * Load of the vdevs (number outstanding I/O requests)
>>   * The locality of last queued I/O vs the new I/O request.
>>
>>   Within the locality calculation additional knowledge about the underlying vdev
>>   is considered such as; is the device backing the vdev a rotating media device.
>>
>>   This results in performance increases across the board as well as significant
>>   increases for predominantly streaming loads and for configurations which don't
>>   have evenly performing devices.
>>
>>   The following are results from a setup with 3 Way Mirror with 2 x HD's and
>>   1 x SSD from a basic test running multiple parrallel dd's.
>>
>>   With pre-fetch disabled (vfs.zfs.prefetch_disable=1):
>>
>>   == Stripe Balanced (default) ==
>>   Read 15360MB using bs: 1048576, readers: 3, took 161 seconds @ 95 MB/s
>>   == Load Balanced (zfslinux) ==
>>   Read 15360MB using bs: 1048576, readers: 3, took 297 seconds @ 51 MB/s
>>   == Load Balanced (locality freebsd) ==
>>   Read 15360MB using bs: 1048576, readers: 3, took 54 seconds @ 284 MB/s
>>
>>   With pre-fetch enabled (vfs.zfs.prefetch_disable=0):
>>
>>   == Stripe Balanced (default) ==
>>   Read 15360MB using bs: 1048576, readers: 3, took 91 seconds @ 168 MB/s
>>   == Load Balanced (zfslinux) ==
>>   Read 15360MB using bs: 1048576, readers: 3, took 108 seconds @ 142 MB/s
>>   == Load Balanced (locality freebsd) ==
>>   Read 15360MB using bs: 1048576, readers: 3, took 48 seconds @ 320 MB/s
>>
>>   In addition to the performance changes the code was also restructured, with
>>   the help of Justin Gibbs, to provide a more logical flow which also ensures
>>   vdevs loads are only calculated from the set of valid candidates.
>>
>>   The following additional sysctls where added to allow the administrator
>>   to tune the behaviour of the load algorithm:
>>   * vfs.zfs.vdev.mirror.rotating_inc
>>   * vfs.zfs.vdev.mirror.rotating_seek_inc
>>   * vfs.zfs.vdev.mirror.rotating_seek_offset
>>   * vfs.zfs.vdev.mirror.non_rotating_inc
>>   * vfs.zfs.vdev.mirror.non_rotating_seek_inc
>>
>>   These changes where based on work started by the zfsonlinux developers:
>>   https://github.com/zfsonlinux/zfs/pull/1487
>>
>>   Reviewed by: gibbs, mav, will
>>   MFC after: 2 weeks
>>   Sponsored by: Multiplay
>
> No MFC yet.

Grr not seen any MFC reminder :( Will send a request to re@ for permission
along with r260713.

    Regards
    Steve