Very low disk performance on 5.x

[Jonathan Noack]

On 05/02/05 23:37, Petri Helenius wrote:
> Robert Watson wrote:
>> The next thing that would be quite nice to measure is the rate of I/O 
>> transactions per second we can get to the disk using the disk device 
>> directly, with a minimal transaction size.  I have a vague 
>> recollection that you have to be careful in Linux because their 
>> character device nodes for disk devices are buffered, and you really 
>> want unbuffered I/O.
> 
> I noticed that changing vfs.read_max from the default 8 to 16 has a 
> dramatic effect on sequential read performance. Increasing it further 
> did not have measurable effect.

Wow.  Setting vfs.read_max=16 resulted in a 50+% performance boost when 
sequentially reading from my 4-disk RAID5 (ida(4)) via the filesystem. 
Write performance was unchanged.  Raw device read performance was also 
unchanged (~40 MB/s).  With vfs.read_max>8, reading through the 
filesystem is *faster* than reading from the raw device!

Note that these tests were run in multi-user on an idle 5.4-RC4 machine 
with 1 GB of RAM and quad P3 Xeons.  The machine is currently using ULE. 
  These tests were repeated 3 times with results in the +-0.1MB/s range.

$ sysctl vfs.read_max
vfs.read_max: 8
$ /usr/bin/time -h dd of=/dev/null if=/usr/testfile bs=64k count=100000
100000+0 records in
100000+0 records out
6553600000 bytes transferred in 185.571466 secs (35315774 bytes/sec)
         3m5.58s real            0.16s user              1m0.89s sys
$ sysctl vfs.read_max=16
vfs.read_max: 8 -> 16
$ /usr/bin/time -h dd of=/dev/null if=/usr/testfile bs=64k count=100000
100000+0 records in
100000+0 records out
6553600000 bytes transferred in 118.846665 secs (55143323 bytes/sec)
         1m58.85s real           0.21s user              1m3.25s sys
$ sysctl vfs.read_max=24
vfs.read_max: 16 -> 24
$ /usr/bin/time -h dd of=/dev/null if=/usr/testfile bs=64k count=100000
100000+0 records in
100000+0 records out
6553600000 bytes transferred in 107.360600 secs (61042878 bytes/sec)
         1m47.36s real           0.14s user              1m3.27s sys
$ sysctl vfs.read_max=32
vfs.read_max: 24 -> 32
$ /usr/bin/time -h dd of=/dev/null if=/usr/testfile bs=64k count=100000
100000+0 records in
100000+0 records out
6553600000 bytes transferred in 102.288822 secs (64069562 bytes/sec)
         1m42.29s real           0.20s user              1m4.52s sys

-- 
Jonathan Noack | noackjr at alumni.rice.edu | OpenPGP: 0x991D8195
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 187 bytes
Desc: OpenPGP digital signature
Url : http://lists.freebsd.org/pipermail/freebsd-performance/attachments/20050503/50e2c1c1/signature.bin