gstripe performance scaling with many disks

[Vasil Dimov]

Hi,

Thank you very much for your answers!

Here is what further tests showed:

First of all I switched from dd to raidtest. I had to tune raidtest in
order to use it, see ports/107311. I am testing only reading with 8
concurrent processes (raidtest test -n 8).

 2 disks:  2103646 b/s (exp  2107584), avg disk load: 1051823 ( 99.8%)
 3 disks:  3134534 b/s (exp  3161376), avg disk load: 1044844 ( 99.1%)
 4 disks:  4153974 b/s (exp  4215168), avg disk load: 1038493 ( 98.5%)
 5 disks:  5199917 b/s (exp  5268960), avg disk load: 1039983 ( 98.6%)
 6 disks:  6141678 b/s (exp  6322752), avg disk load: 1023613 ( 97.1%)
 7 disks:  7193116 b/s (exp  7376544), avg disk load: 1027588 ( 97.5%)
 8 disks:  8219609 b/s (exp  8430336), avg disk load: 1027451 ( 97.5%)
 9 disks:  9080762 b/s (exp  9484128), avg disk load: 1008973 ( 95.7%)
10 disks: 10241349 b/s (exp 10537920), avg disk load: 1024134 ( 97.1%)
11 disks: 11077983 b/s (exp 11591712), avg disk load: 1007089 ( 95.5%)
12 disks: 11851009 b/s (exp 12645504), avg disk load:  987584 ( 93.7%)
13 disks: 12663548 b/s (exp 13699296), avg disk load:  974119 ( 92.4%)
14 disks: 13821213 b/s (exp 14753088), avg disk load:  987229 ( 93.6%)
15 disks: 14283895 b/s (exp 15806880), avg disk load:  952259 ( 90.3%)
16 disks: 15057168 b/s (exp 16860672), avg disk load:  941073 ( 89.3%)
17 disks: 16171889 b/s (exp 17914464), avg disk load:  951287 ( 90.2%)

It shows the same tendency as with dd(1).
Changing vfs.read_max from 8 to 32 does not produce different results.

This are gstat screen shots during the test:

1 drive:
dT: 0.501s  w: 0.500s
 L(q)  ops/s    r/s   kBps   ms/r    w/s   kBps   ms/w   %busy Name
    8     12     12   1003  548.1      0      0    0.0   96.6| ggate100

8 drives:
dT: 0.501s  w: 0.500s
 L(q)  ops/s    r/s   kBps   ms/r    w/s   kBps   ms/w   %busy Name
    6    116    116   1028   63.4      0      0    0.0   97.6| ggate100
    2    118    118   1080   50.6      0      0    0.0  102.0| ggate101
    4    114    114   1042   45.9      0      0    0.0  100.8| ggate102
    4    110    110    982   39.3      0      0    0.0   96.0| ggate103
    4    116    116   1027   43.7      0      0    0.0   98.6| ggate104
    6    116    116   1056   58.4      0      0    0.0  101.6| ggate105
    8    124    124   1029   60.0      0      0    0.0   98.0| ggate107
    7    122    122   1051   61.9      0      0    0.0  106.6| ggate106
    8    130    130   8230   62.8      0      0    0.0   99.6| stripe/stest

17 drives:
dT: 0.563s  w: 0.500s
 L(q)  ops/s    r/s   kBps   ms/r    w/s   kBps   ms/w   %busy Name
    5    185    178    848   18.9      0      0    0.0   88.8| ggate100
    4    183    176    831   19.8      0      0    0.0   87.9| ggate101
    4    181    174    822   18.8      0      0    0.0   87.0| ggate102
    4    165    165    801   17.7      0      0    0.0   86.6| ggate103
    2    167    167    820   17.3      0      0    0.0   85.0| ggate104
    1    176    176    840   18.8      0      0    0.0   86.4| ggate105
    4    167    167    812   23.5      0      0    0.0   85.5| ggate107
    3    167    167    802   19.9      0      0    0.0   85.9| ggate108
    1    172    172    841   19.9      0      0    0.0   86.5| ggate109
    2    170    170    852   28.5      0      0    0.0   87.4| ggate110
    4    172    172    847   21.4      0      0    0.0   87.1| ggate111
    4    174    174    838   22.3      0      0    0.0   87.0| ggate112
    5    163    163    789   16.2      0      0    0.0   86.8| ggate113
    4    162    162    802   15.7      0      0    0.0   86.4| ggate114
    3    162    162    823   17.0      0      0    0.0   86.2| ggate115
    3    170    170    825   18.2      0      0    0.0   86.6| ggate116
    5    169    169    810   25.7      0      0    0.0   86.1| ggate106
    8    245    238  13573   27.5      0      0    0.0   90.1| stripe/stest

I have not tried geom_cache...

-- 
Vasil Dimov
gro.DSBeerF at dv
%
If the code and the comments disagree, then both are probably wrong.
                -- Norm Schryer
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 155 bytes
Desc: not available
Url : http://lists.freebsd.org/pipermail/freebsd-geom/attachments/20061229/4a2f8f2b/attachment.pgp