High load averages on a 5.3-RELEASE

Wed Sep 14 13:11:50 PDT 2005

Hi people, I've also posted this message to freebsd-performance, with no
answer.
I have a mail server running FreeBSD 5.3-RELEASE. It's a dual Xeon with 4gb
of ram.
The server is running Apache (serving Horde) , Postfix, Courier imapd w/SSL,
Amavisd. Also a Postgresql as Horde's storage.
The mail storage is accessed via NFS (this server is the nfs client).
The problem is that it's experiencing a really high load average (see the
top -S and vmstat -systat ) while the CPU's are relatively idle. I think
it's a IO problem
(nfs?), but I cant figure out what exactly is causing this.

last pid: 96776;  load averages: 13.60, 13.94, 18.26               up
22+01:45:44  17:08:08
1166 processes:8 running, 1087 sleeping, 71 waiting
CPU states: 12.3% user,  0.0% nice, 17.8% system,  2.1% interrupt, 67.8%
idle
Mem: 1637M Active, 1638M Inact, 320M Wired, 192M Cache, 112M Buf, 28M Free
Swap: 2023M Total, 216K Used, 2023M Free

  PID USERNAME      PRI NICE   SIZE    RES STATE  C   TIME   WCPU    CPU
COMMAND
   11 root          132    0     0K    12K RUN    3 492.1H 82.13% 82.13%
idle: cpu3
   14 root          132    0     0K    12K RUN    0 487.0H 79.83% 79.83%
idle: cpu0
   12 root          132    0     0K    12K RUN    2 487.0H 78.37% 78.37%
idle: cpu2
   13 root          132    0     0K    12K CPU1   1 463.9H 71.63% 71.63%
idle: cpu1
   24 root          -64 -183     0K    12K CPU0   0 154:50  2.29%  2.29%
irq11: ohci0
79636 www             4    0 33532K 22264K sbwait 2   0:08  1.27%  1.27%
httpd
   78 root          -44 -163     0K    12K WAIT   1 371:20  1.12%  1.12%
swi1: net
95810 root            5    0  1844K  1432K ttyin  1   0:03  1.03%  1.03%
systat
59508 root            5    0  1836K  1336K ttyin  3   5:56  0.98%  0.98%
systat
   79 root          -28 -147     0K    12K WAIT   1 171:24  0.93%  0.93%
swi5: clock sio
72854 www            96    0 34068K 22800K select 1   0:11  0.93%  0.93%
httpd
90730 www            20    0 29720K 18452K lockf  0   0:02  0.93%  0.93%
httpd
77968 www             4    0 36816K 25556K sbwait 2   0:09  0.88%  0.88%
httpd
78081 www            20    0 31916K 20640K lockf  1   0:07  0.83%  0.83%
httpd
96143 postfix        96    0  3180K  2284K select 1   0:02  0.78%  0.78%
smtpd
70626 www            20    0 33264K 21992K lockf  3   0:08  0.73%  0.73%
httpd
71231 root           96    0  2572K  1696K select 1   0:03  0.68%  0.68%
couriertls
87567 www             4    0 33276K 22040K sbwait 3   0:04  0.63%  0.63%
httpd
78109 www            20    0 34228K 22968K lockf  1   0:08  0.49%  0.49%
httpd
71240 mfx            96    0  2596K  1532K select 0   0:02  0.49%  0.49%
imapd
95217 vscan          20    0 20068K 18088K lockf  3   0:01  0.49%  0.49%
perl
79635 www            20    0 34160K 22884K lockf  3   0:05  0.44%  0.44%
httpd
75988 pgsql           4    0 20436K 14468K sbwait 2   0:02  0.44%  0.44%
postgres
96754 postfix         4    0  3180K  2284K select 2   0:00  4.61%  0.44%
smtpd
79861 pgsql           4    0 20436K 14332K sbwait 0   0:01  0.39%  0.39%
postgres
90802 www            20    0 33272K 22032K lockf  1   0:03  0.34%  0.34%
httpd
96763 postfix         4    0  3160K  2272K select 0   0:00  7.00%  0.34%
smtpd
96756 postfix         4    0  3088K  2204K select 3   0:00  3.08%  0.29%
smtp
85422 www            20    0 34168K 22892K lockf  3   0:04  0.24%  0.24%
httpd
90750 www             4    0 33284K 22024K sbwait 1   0:02  0.24%  0.24%
httpd

    5 users    Load 11.65 13.51 18.06                  14 sep 17:08

(this is ugly, sorry):

Mem:KB    REAL            VIRTUAL                     VN PAGER  SWAP PAGER
        Tot   Share      Tot    Share    Free         in  out     in  out
Act 1624872   13936  3218536    27392  226332 count
All 3864628   39728  2312968   378336         pages
                                                                 Interrupts
Proc:r  p  d  s  w    Csw  Trp  Sys  Int  Sof  Flt    162 cow    1138 total
     1     3 1k      5797  599 6050 1793  133  688 326888 wire        1:
atkb
                                                  1673300 act         6:
fdc0
 6.1%Sys   2.2%Intr  8.7%User  0.0%Nice 83.0%Idl  1681452 inact   128 8: rtc
|    |    |    |    |    |    |    |    |    |     196908 cache   166 11:
ohc
===+>>>>>                                           29424 free        13:
npx
                                                          daefr       14:
ata
Namei         Name-cache    Dir-cache                 384 prcfr    18 15:
ata
    Calls     hits    %     hits    %                     react    18 16:
cis
    27907    27560   99                                   pdwak   541 26:
bge
                                      252 zfod            pdpgs   167 29:
bge
Disks   da0 pass0                     248 ofod            intrn       30:
ahc
KB/t  13.57  0.00                      98 %slo-z   114464 buf         31:
ahc
tps      18     0                     645 tfree       178 dirty   100 0: clk
MB/s   0.24  0.00                                  100000 desiredvnodes
% busy    6     0                                   91116 numvnodes
                                                    46875 freevnodes

After searching lists and documentation, I'm not sure what's
causing the problem.
Suggestions ?

Thanks in advance,
Mariano.

-- 
Lust, ein paar Euro nebenbei zu verdienen? Ohne Kosten, ohne Risiko!
Satte Provisionen für GMX Partner: http://www.gmx.net/de/go/partner