7.0-Release and 3ware 9550SXU w/BBU - horrible write performance

[Ivan Voras]

Erik Stian Tefre wrote:

> It was completely idle. Changing vfs.read_max to 80 triples the
> sequential read performance, see bonnie++ output below (run on the same
> box, nothing changed except vfs.read_max).

It might be that 3ware is specially pessimized by FreeBSD chopping IO
into 64K blocks. But vfs.read_max doesn't change that so it's maybe not it.

> bonnie++ with vfs_max=80:
> Version 1.93d       ------Sequential Output------ --Sequential Input-
> --Random-
> Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
> --Seeks--
> Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP
> /sec %CP
> xxxxxxxxxxxxxxxx 1G   285  83 88193  28 26858   6   690  89 218642  59
> 406.5   5
> Latency             44353us     557ms     542ms   90795us     209ms 361ms
> Version 1.93d       ------Sequential Create------ --------Random
> Create--------
> xxxxxxxxxxxxxxxxxxx -Create-- --Read--- -Delete-- -Create-- --Read---
> -Delete--
>               files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
> /sec %CP
>                  16  8204  19 +++++ +++ 22163  36  9925  25 +++++ +++
> +++++ +++
> Latency               585ms     571us    3844us     257ms   24015us 768us


Are these numbers typical for 3ware's controllers? I still think
something's bad about your setup, see the following performance results
on a 3-drive RAID5 on Dell PERC5:

Version 1.93d       ------Sequential Output------ --Sequential Input-
--Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP
/sec %CP
xxxxx.xxx.xx  4000M   298  99 92319  25 30729  13   440  88 121370  25
533.1  29
Latency             28140us     711ms     430ms     528ms   74013us
225ms
Version 1.93d       ------Sequential Create------ --------Random
Create--------
xxxxx.xxx.xx        -Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
/sec %CP
                 30 18782  85 50106  99 45405  99 16481  58 60185  99
49638  99
Latency               259ms   21091us     130us   27529us      50us
 75us

I consider this ok, since (for the simple case of READing) the 3-drive
RAID5 array has the performance of a 2-drive striped array.

Your CPU usage is quite high (59% on sequential block input, if I'm
reading it correctly) - are you limited by your CPU?



-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: OpenPGP digital signature
Url : http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20080305/ec4fd57a/signature.pgp