gmirror 'load' algorithm (Was: Re: siis/atacam/ata/gmirror 8.0-BETA3
disk performance)
Alexander Motin
mav at FreeBSD.org
Thu Sep 3 18:13:58 UTC 2009
Emil Mikulic wrote:
> On Wed, Sep 02, 2009 at 05:51:35PM +0300, Alexander Motin wrote:
>> To completely load gmirror on read operations, you may need to run
>> two dd's same time. Also make sure, that your gmirror runs in
>> round-robin mode. Default split mode, which should help with linear
>> read, is IMHO ineffective, at least with default MAXPHYS and slice
>> values.
>
> On that note, there is an excellent patch in this PR which improves
> the way gmirror schedules read requests to different disks:
>
> http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/113885
>
> Could someone please commit this?
>
> With this patch and a two-way mirror, I can run two linear scans of
> different files in parallel and get almost perfect scaling. (result:
> this approximately halves the wall-clock time it takes to do a backup of
> some fat VM images)
>
> IIRC, without the patch it's faster to run them sequentially. :(
I have played a bit with this patch on 4-disk mirror. It works better
then original algorithm, but still not perfect.
1. I have managed situation with 4 read streams when 3 drives were busy,
while forth one was completely idle. gmirror prefer constantly seek one
of drives on short distances, but not to use idle drive, because it's
heads were few gigabytes away from that point.
IMHO request locality priority should be made almost equal for any
nonzero distances. As we can see with split mode, even small gaps
between requests can significantly reduce drive performance. So I think
it is not so important if data are 100MB or 500GB away from current head
position. It is perfect case when requests are completely sequential.
But everything beyond few megabytes from current position just won't fit
drive cache.
2. IMHO it would be much better to use averaged request queue depth as
load measure, instead of last request submit time. Request submit time
works fine only for equal requests, equal drives and serialized load,
but it is actually the case where complicated load balancing is just not
needed. The fact that some drive just got request does not mean
anything, if some another one got 50 requests one second ago and still
processes them.
--
Alexander Motin
More information about the freebsd-current
mailing list