Performance of disk i/o with 3ware

[Nico -telmich- Schottelius]

Manjunath R Gowda [Tue, Nov 06, 2007 at 09:54:09AM -0800]:
> On 11/6/07, Nico -telmich- Schottelius <
> nico-freebsd-performance at schottelius.org> wrote:
> >
> > Hello!
> >
> > I've the problem that sometimes there are many disk waiting processes
> > (sysctl -n vm.vmtotal), but systat -vmstat shows da0 and da1 busy with
> > 0-10%.
> 
> I woudn't worry about how many are sleeping. But, How long they sleep would
> be interesting data.

And also why they sleep. Having about 700 processes in sleep state plus
400 processes in diskwait state makes me wonder somehow.

There are also some processes from cron, which are older and also in
state "I" (as shown by ps).

> > I guess that the disk i/o is at about 100%, but wondering why I see
> > those strange values.
> 
> Why do you think it should be 100%? Are you running any disk I/O intense
> application?

Because it's the well known bottleneck in that server:

- it has 4 cpus, which are about 30-50% idle
- it has 4 GiB ram, of which 2GiB is mostly inactive
- it has 4 10k rpm hds in 2 raid1 disk arrays, one for
  the mailboxes + root and one for the qmail-queue

Normally systat -vmstat shows 80-100% busy state on the disk arrays,
but currently it's at about 10%, which cannot be right.

I think I'll have a look closer look at the patch we used from CVS to patch the
3ware twa driver.

> > Anyone an idea,
> >   a) why systat -vmstat shows so small busy values?
> >   b) how to debug it further?
> 
> You can try experimenting  with diffrent I/O loads to make sure that there
> is a problem before start debugging it.

There is a big problem, that even results in taking about 30 seconds
until the '220' messages comes from qmail when connecting via telnet
from outside to it.

To summarise, what runs on the server:

- qmail + patches => has /var/qmail/queue on it's own disk-array
- courier imapd
- vpopmail (pop3 auth)
- mysql (needed by vpopmail)
- clamd
- spamassassin
- Webmail (horde on apache)

So it's mainly a customer mailserver, with nothing special installed,
which may have some load, but not in the way it's not explainable.

So I'm still hoping somebody with similar problems reads this mail ;-)

Sincerly

Nico

-- 
Think about Free and Open Source Software (FOSS).
http://nico.schottelius.org/documentations/foss/the-term-foss/

PGP: BFE4 C736 ABE5 406F 8F42  F7CF B8BE F92A 9885 188C
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://lists.freebsd.org/pipermail/freebsd-performance/attachments/20071106/b3b429e4/attachment.pgp