"Load Balancing": How Busy are the servers?

[Francisco Reyes]

Marc G. Fournier writes:

> 1. What variables on a server should be monitored to determine how busy a 
> server is? 

I am a fairly new sysadmin.. who inheritted nearly 20 machines, so take my 
comments with a gain of salt. Before that the most I ever had was 7, mostly 
DB, FreeBSD machines :-) 
.. and.. Hi Marc. :)

I think it comes down to primarily 3 factors
* RAM
* CPU
* DISK

If you are hitting Swap, you are either running too many programs/services 
or too many users.

Same for CPU

Disk are different in that the same number of disks can perform different 
based on what raid controller and what type of RAID.

I use top and load average to determine if a machine is up to capacity in 
memory/cpu.

I use vmstat to determine if the disk subsystem is falling behind.

BIG NOTE: 
The one thing that I have yet to really pay much attention is the network 
performance. Fortunately we just hired someone who has significantly more 
experience on that area. :-)


> 2. Are there any tools that I can run to give me a point in time "summary" 
> of how busy a server is based on these several factors?

I think there are lots of tools. Some vary from SNMP capture/graphing, to 
custom made tools done in-house. I think it's a combination of how difficult 
it is to setup vs what you need to monitor. 

At work we are just starting to roll out an SNMP tool. The new hire is 
leading the effort so I am not very familiar with the setups.. the one thing 
I see so far is that ultimately, there usually are things that one needs to 
monitor that is unique to your organization and you need to either integrate 
a program into the tool or do your own independant monitoring of that 
particular resource.

I think the ISP list may be a good resource since the needs of the average 
user are different from ISPs/companies with numerous machines. 

> Basically, I'd like to keep track of multiple servers and be able to say 
> "this server is running >75% of capacity, time to upgrade or move things 
> off of it" ... if its possible ... ?

In my opinion, for the most part, the answer is yes. The problem is usually 
how long it's going to take you to setup the environment to monitor the 
servers.

The program we went with was chosen because the new hire was familiar with 
it, but a search on the archives for "monitoring tools" will give you a long 
list of programs and opinions of which are easier.

If I had the time, I think I would likely write my own tool. This way I will 
be able to measure exactly what I want. Right now I thik we will cover most 
basics with the tool we are going with, but will need to still do our own 
custom apps to monitor a number of resources and metrics.