massive load average spikes

[Chuck Swiger]

Hi--

On Aug 11, 2010, at 10:04 AM, markham breitbach wrote:
> I am running into an issue where I am seeing load average on a server suddenly jump from
> nominal values around 0.5 to anywhere from 10 up over 70 in under 1 second.  This does not
> seem to be related to CPU overload, and LA immediately begins to fall back again to
> nominal.  This does not seem to happen with any regular frequency, and can happen several
> times an hour or not for hours.
[ ... ]
> Can anyone suggest what may be causing this or how to track that down?

>From the (limited) available data, I'd imagine someone is doing wardialling of your mail service to try common username/password combinations and break in.  Especially if they are connecting via POP3S / IMAPS ports and doing SSL negotiation, there's a very high burst of CPU load, as imap or pop daemons get forked to handle the requests, then quit immediately afterwards when the login attempt fails.  You won't see much change in memory loading unless they do get a valid login since the Dovecot daemons are already resident & there's no real I/O made to disk until it looks up a real user's mail.

Looking at tcpdump for new connection requests or checking the Dovecot mail logs for a slew of attempted logins for invalid users, and correlating with your load spikes would be a way of checking on this theory....

Regards,
-- 
-Chuck