High load average mail server 5.3-RELEASE

[Eric Anderson]

Francisco Reyes wrote:
> On Thu, 22 Sep 2005, Eric Anderson wrote:
> 
>> Also, if it is an NFS server, one should check the cpu times on the 
>> nfsd processes.  I've found that many times there aren't enough nfsd 
>> processes to take the load from many clients.  Increasing the number 
>> (double it) often helps this.  The max in 5.3 is 20, but you can 
>> easily change it and get around it.
> 
> 
> What is the parameter to change the number of nfsd processes?

Use the -n flag to nfsd, so in /etc/rc.conf:

nfs_server_flags="-u -t -n 1024"

Yep, that's right, I have mine set to 1024.   How I got to that number 
is simply by watching the cpu time on my nfsd processes.  I do this:

ps -auxw | grep nfsd  | head -n 20
Which would reveal something like:
root       410  4.2  0.0  1236   732  ??  S    13Sep05 1400:55.70 nfsd: 
server (nfsd)
root       411  0.0  0.0  1236   732  ??  S    13Sep05 320:48.27 nfsd: 
server (nfsd)
root       409  0.0  0.0  1352   924  ??  Is   13Sep05   0:00.47 nfsd: 
master (nfsd)
root       412  0.0  0.0  1236   732  ??  S    13Sep05  83:29.31 nfsd: 
server (nfsd)
root       413  0.0  0.0  1236   732  ??  S    13Sep05  39:18.94 nfsd: 
server (nfsd)
root       414  0.0  0.0  1236   732  ??  S    13Sep05  25:16.79 nfsd: 
server (nfsd)
root       416  0.0  0.0  1236   732  ??  S    13Sep05  19:39.53 nfsd: 
server (nfsd)
root       417  0.0  0.0  1236   732  ??  S    13Sep05  15:23.14 nfsd: 
server (nfsd)
root       418  0.0  0.0  1236   732  ??  S    13Sep05  13:05.99 nfsd: 
server (nfsd)
root       419  0.0  0.0  1236   732  ??  S    13Sep05  11:05.15 nfsd: 
server (nfsd)
root       420  0.0  0.0  1236   732  ??  S    13Sep05   9:59.39 nfsd: 
server (nfsd)
root       421  0.0  0.0  1236   732  ??  S    13Sep05   8:39.97 nfsd: 
server (nfsd)
root       422  0.0  0.0  1236   732  ??  S    13Sep05   7:34.38 nfsd: 
server (nfsd)
root       423  0.0  0.0  1236   732  ??  S    13Sep05   6:56.37 nfsd: 
server (nfsd)
root       424  0.0  0.0  1236   732  ??  S    13Sep05   6:40.99 nfsd: 
server (nfsd)
root       425  0.0  0.0  1236   732  ??  S    13Sep05   6:08.44 nfsd: 
server (nfsd)
root       426  0.0  0.0  1236   732  ??  S    13Sep05   5:40.81 nfsd: 
server (nfsd)
root       427  0.0  0.0  1236   732  ??  S    13Sep05   6:34.98 nfsd: 
server (nfsd)
root       428  0.0  0.0  1236   732  ??  S    13Sep05   4:34.36 nfsd: 
server (nfsd)
root       429  0.0  0.0  1236   732  ??  S    13Sep05   3:17.89 nfsd: 
server (nfsd)


That's on a very fast box, with a fiber channel connected 16 disk 
RAID0+1, with the fastest disks I can buy.  Now, with slower disk back 
end, you'll probably see something more like this:

root       438  0.0  0.1  1372  964  ??  Is   17Jun05   0:03.14 nfsd: 
master (nfsd)
root       439  0.0  0.1  1224  744  ??  S    17Jun05 1010:27.86 nfsd: 
server (nfsd)
root       440  0.0  0.1  1224  744  ??  I    17Jun05 231:17.30 nfsd: 
server (nfsd)
root       441  0.0  0.1  1224  744  ??  I    17Jun05 125:17.14 nfsd: 
server (nfsd)
root       442  0.0  0.1  1224  744  ??  I    17Jun05  76:30.62 nfsd: 
server (nfsd)
root       444  0.0  0.1  1224  744  ??  I    17Jun05  94:18.99 nfsd: 
server (nfsd)
root       445  0.0  0.1  1224  744  ??  I    17Jun05  54:48.31 nfsd: 
server (nfsd)
root       446  0.0  0.1  1224  744  ??  I    17Jun05  63:40.74 nfsd: 
server (nfsd)
root       447  0.0  0.1  1224  744  ??  I    17Jun05  38:29.15 nfsd: 
server (nfsd)
root       448  0.0  0.1  1224  744  ??  I    17Jun05  46:12.29 nfsd: 
server (nfsd)
root       449  0.0  0.1  1224  744  ??  I    17Jun05  15:24.99 nfsd: 
server (nfsd)
root       450  0.0  0.1  1224  744  ??  I    17Jun05  13:20.40 nfsd: 
server (nfsd)
root       451  0.0  0.1  1224  744  ??  I    17Jun05  28:09.07 nfsd: 
server (nfsd)
root       452  0.0  0.1  1224  744  ??  I    17Jun05  11:19.46 nfsd: 
server (nfsd)
root       453  0.0  0.1  1224  744  ??  I    17Jun05  19:34.98 nfsd: 
server (nfsd)
root       454  0.0  0.1  1224  744  ??  I    17Jun05   9:59.46 nfsd: 
server (nfsd)
root       455  0.0  0.1  1224  744  ??  I    17Jun05   8:36.71 nfsd: 
server (nfsd)
root       456  0.0  0.1  1224  744  ??  I    17Jun05   7:28.79 nfsd: 
server (nfsd)
root       457  0.0  0.1  1224  744  ??  I    17Jun05   6:18.29 nfsd: 
server (nfsd)
root       458  0.0  0.1  1224  744  ??  I    17Jun05   5:43.77 nfsd: 
server (nfsd)


See how much run time that 20th process has?  I like to always have a 
couple with zero run time.   So I double the number until I get there, 
then tune back a little.  I've found that a very rough rule of thumb is:

number of nfs clients * 1.8 = nfsd processes

as long as you have the memory for it.  You should consider about:

nfsd processes * 2.5MB = memory desired for all the nfsd's

Eric



-- 
------------------------------------------------------------------------
Eric Anderson        Sr. Systems Administrator        Centaur Technology
Anything that works is better than anything that doesn't.
------------------------------------------------------------------------