3ware 9650SE performance issues

[Nikolas Hagelstein]

Hi,

I am running the 3ware 9650SE(8LPML) in RAID5 with 7x700GB drives
attached to it (~4.2 TB at raid5).
Since i need one huge "storage area" the filesystem resides at the raw
device (i.e. /dev/da0).

For some reasons the raids write performance is very slow (2 to 3MB/s):
33554432 bytes transferred in 14.679426 secs (2285814 bytes/sec) 

The raid is initialized and all disks are ok (double checked that)

I tried several different filesystem and raidcontroller options (e.g.
blocksize,frag, Slicesice etc.) without any success.

Another strange thing: for testing prupose i set the controllerto raid0
mode which resulted in 55MB/s. Though raid0 is faster than raid5 by
principle that anyhow seems really strange to me.

I am not sure if this is a pure hardware related issue since "raw
access" performes quite good:
diskinfo -t /dev/da0
/dev/da0
        512             # sectorsize
        4499932446720   # mediasize in bytes (4.1T)
        8788930560      # mediasize in sectors
        547085          # Cylinders according to firmware.
        255             # Heads according to firmware.
        63              # Sectors according to firmware.
....
Transfer rates:
        outside:       102400 kbytes in   1.052566 sec =    97286
kbytes/sec
        middle:        102400 kbytes in   1.065493 sec =    96106
kbytes/sec
        inside:        102400 kbytes in   1.067073 sec =    95963
kbytes/sec

I attached any needed information:

FreeBSD 6.2-RELEASE

Dmesg snippet(a full version can be find here:http://phpfi.com/203614):
---
3ware device driver for 9000 series storage controllers, version:
3.60.03.005
twa0: <3ware 9000 series Storage Controller> port 0xe800-0xe8ff mem
0xde000000-0xdfffffff,0xbffff000-0xbfffffff irq 16 at device 0.0 on pci4
twa0: [GIANT-LOCKED]
twa0: INFO: (0x15: 0x1300): Controller details:: Model 9650SE-8LPML, 8
ports, Firmware FE9X 3.06.00.005, BIOS BE9X 3.06.00.002

da0 at twa0 bus 0 target 0 lun 0
da0: <AMCC 9650SE-8LP DISK 3.06> Fixed Direct Access SCSI-5 device
da0: 100.000MB/s transfers
da0: 4291470MB (8788930560 512 byte sectors: 255H 63S/T 547085C)
---

---
tunefs -p /dev/da0
tunefs: ACLs: (-a)                                         disabled
tunefs: MAC multilabel: (-l)                               disabled
tunefs: soft updates: (-n)                                 disabled
tunefs: gjournal: (-J)                                     disabled
tunefs: maximum blocks per file in a cylinder group: (-e)  2048
tunefs: average file size: (-f)                            16384
tunefs: average number of files in a directory: (-s)       64
tunefs: minimum percentage of free space: (-m)             8%
tunefs: optimization preference: (-o)                      time
tunefs: volume label: (-L)
---


---

Ctl   Model        Ports   Drives   Units   NotOpt   RRate   VRate   BBU
------------------------------------------------------------------------
c0    9650SE-8LPML 8       7        1       0        1       1       -

//backup> /c0 show

Unit  UnitType  Status         %RCmpl  %V/I/M  Stripe  Size(GB)  Cache
AVrfy
------------------------------------------------------------------------
------
u0    RAID-5    OK             -       -       256K    4190.89   OFF
OFF

Port   Status           Unit   Size        Blocks        Serial
---------------------------------------------------------------
p0     OK               u0     698.63 GB   1465149168    3QD0Q4N6
p1     OK               u0     698.63 GB   1465149168    3QD0RYEP
p2     OK               u0     698.63 GB   1465149168    3QD0HEZM
p3     OK               u0     698.63 GB   1465149168    3QD0PZK7
p4     OK               u0     698.63 GB   1465149168    3QD07Q7G
p5     OK               u0     698.63 GB   1465149168    3QD0PX0J
p6     OK               u0     698.63 GB   1465149168    3QD0M5VQ
p7     NOT-PRESENT      -      -           -             -
---

Cheers,
Nikolas