raid3 is slow

[Szabó Péter]

I stop the copy

#diskinfo -vt /dev/raid3/nmivol
/dev/raid3/nmivol
        2048            # sectorsize
        638906845184    # mediasize in bytes (595G)
        311966233       # mediasize in sectors

Seek times:
        Full stroke:      250 iter in  11.486148 sec =   45.945 msec
        Half stroke:      250 iter in   8.818982 sec =   35.276 msec
        Quarter stroke:   500 iter in  15.365682 sec =   30.731 msec
        Short forward:    400 iter in   6.013585 sec =   15.034 msec
        Short backward:   400 iter in  10.356176 sec =   25.890 msec
        Seq outer:       2048 iter in   0.803544 sec =    0.392 msec
        Seq inner:       2048 iter in   0.804926 sec =    0.393 msec
Transfer rates:
        outside:       102400 kbytes in   1.179933 sec =    86785 kbytes/sec
        middle:        102400 kbytes in   1.148816 sec =    89135 kbytes/sec
        inside:        102400 kbytes in   1.171987 sec =    87373 kbytes/sec

#diskinfo -vt /dev/label/nmivol1
/dev/label/nmivol1
        512             # sectorsize
        159726711808    # mediasize in bytes (149G)
        311966234       # mediasize in sectors
        309490          # Cylinders according to firmware.
        16              # Heads according to firmware.
        63              # Sectors according to firmware.

Seek times:
        Full stroke:      250 iter in   5.540328 sec =   22.161 msec
        Half stroke:      250 iter in   3.955182 sec =   15.821 msec
        Quarter stroke:   500 iter in   6.648877 sec =   13.298 msec
        Short forward:    400 iter in   2.203517 sec =    5.509 msec
        Short backward:   400 iter in   1.960238 sec =    4.901 msec
        Seq outer:       2048 iter in   0.288225 sec =    0.141 msec
        Seq inner:       2048 iter in   0.292780 sec =    0.143 msec
Transfer rates:
        outside:       102400 kbytes in   1.839020 sec =    55682 kbytes/sec
        middle:        102400 kbytes in   2.112483 sec =    48474 kbytes/sec
        inside:        102400 kbytes in   3.416896 sec =    29969 kbytes/sec

# atacontrol mode ad0
current mode = UDMA100
# atacontrol mode ad1
current mode = UDMA100
# atacontrol mode ad2
current mode = UDMA100
# atacontrol mode ad3
current mode = UDMA100
# atacontrol mode ad6
current mode = SATA150

gstripe may help, and extends disk space, but less secure.

#dd if=/dev/zero of=/mnt/new/x.x bs=1m count=128
128+0 records in
128+0 records out
134217728 bytes transferred in 11.410957 secs (11762180 bytes/sec)


Matyee

----- Original Message ----- 
From: "Eric Anderson" <anderson at freebsd.org>


On 03/28/07 07:25, Szabó Péter wrote:
> Yes, consumers are the disks.

[..snip..]

> #systat -vmstat
>
>     4 users    Load  1.62  1.75  1.82                  Mar 28 14:24
>
> Mem:KB    REAL            VIRTUAL                     VN PAGER  SWAP PAGER
>         Tot   Share      Tot    Share    Free         in  out     in  out
> Act   66488    6968   150740    11396   31172 count
> All 1017076   10160975701764    16256         pages
>                                                         1 zfod 
> Interrupts
> Proc:r  p  d  s  w    Csw  Trp  Sys  Int  Sof  Flt        cow    4066 
> total
>            1 60      9820  68312106 4237  752    1 195968 wire   1002 0: 
> clk
>                                                     79088 act     128 8: 
> rtc
> 89.2%Sys   3.0%Intr  3.3%User  0.0%Nice  4.5%Idl   714076 inact    10 10: 
> fxp
> |    |    |    |    |    |    |    |    |    |      29068 cache  1780 11: 
> ata
> =============================================+>>     2104 free    573 14: 
> ata
>                                                        12 daefr   573 15: 
> ata
> Namei         Name-cache    Dir-cache                     prcfr
>     Calls     hits    %     hits    %                     react
>        11       10   91                                 1 pdwake
>                                                      4045 pdpgs
> Disks   ad0   ad1   ad2   ad3   ad6                       intrn
> KB/t   8.42  8.42  8.42  8.42  5.67                113904 buf
> tps     287   287   287   287  1780                   133 dirtybuf
> MB/s   2.36  2.36  2.36  2.36  9.85                 69984 desiredvnodes
> % busy   25    26    24    26    99                 21586 numvnodes
>                                                     17495 freevnodes
>
> I use mc to copy.
>
> Matyee


Can you send the output of:

diskinfo -vt /dev/raid3/nmivol
diskinfo -vt /dev/label/nmivol1

Thanks
Eric



>> Szabó Péter wrote:
>>
>>> I have a raid3 setup with 5 consumers. it seems everything is all right, 
>>> but riad is very slow imho, but i don't have any >reference.
>> You need to give more accurate information. What are the "consumers" you
>> mention? Maybe you mean the disks/components of the RAID?
>>
>>> Now i copy from raid3 to a single hdd 75GB data. Nothing else run, 
>>> except ssh, and ftp. Before the copy my load is >0.1, after i star the 
>>> copy the load is 2.92 or higher, and the machine is very slow. I think 
>>> this is not normal, or is it?
>>> Copy how? By "cp" from shell?
>>> My config is:
>>> AMD Barton 2500+ with 1GB memory.
>>>
>>> I stop the copy, load is 0.05, and start again, after a few seconds load 
>>> is 2.02.
>
>> Due to the way raid3 works, it's by definition more CPU-intensive than
>> most other RAID levels, but it shouldn't be even remotely slow as that.
>> Also, since your load is > 2, what are the other CPU intensive tasks?
>