sudden peak in load average

[Kris Kennaway]

Zbigniew Szalbot wrote:
> Hello,
> 
>>From the MRTG data I can see that suddenly on a currently not very
> busy machine the load averege went over 15 or more. This happened
> around 10 in the morning. Not many entries in httpd access log, smtp
> server was not too much loaded (at that time it generally produces a
> load average of about 1).
> 
> But this time:
> 2008-02-27 10:03:36 1JUICO-0002Bs-23 no immediate delivery: load average 15.88
> 
> Where can I look for more information about what happened. The server
> was rebooted early in the morning so its swap consumption was almost
> non-existent. Around 10 it suddenly consumed probably around 200MB of
> swap, now it stands at 135MB which is pretty normal with this machine
> (it only has 512 RAM). SWAP usage is normal with this machine but it
> usually is a matter of 2-3 days before it gradually takes over 10-13%
> of swap space.
> 
> I tried by looking at:
> http access log
> auth.log
> maillog
> messages
> 
> Nothing in there indicating an outburst of sudden activities.
> 
> heh - I now think I may be wrong. The load average did not necessarily
> produce so much swap consumption so fast. At 10:13 I run a cron job
> optimising all mysql tables. So maybe in fact the swap was used by
> mysql operations, although I am still interested to now what casued
> load average to go above 15 at 10:03.
> 
> 10:00AM  up  3:14, 0 users, load averages: 0.40, 0.13, 0.05
> 10:01AM  up  3:15, 0 users, load averages: 0.23, 0.14, 0.05
> 10:02AM  up  3:16, 0 users, load averages: 0.08, 0.11, 0.05
> 10:03AM  up  3:17, 0 users, load averages: 11.54, 3.09, 1.15
> 10:04AM  up  3:18, 0 users, load averages: 13.26, 5.69, 2.28
> 10:05AM  up  3:19, 0 users, load averages: 4.98, 4.69, 2.14
> 10:06AM  up  3:20, 0 users, load averages: 1.79, 3.80, 1.99
> 
> 
> Thanks for any pointers.

No way to know.  You'll have to set up more detailed logging, e.g. a 
script that runs ps if load is over a certain limit.

Kris