Too Much Context Switching?

[Kris Kennaway]

alex at schnarff.com wrote:
> I'm the webmaster for www.marssociety.org, which is a FreeBSD 
> 6.2-RELEASE box running on a dual-core AMD Opteron setup with 4GB of 
> RAM. The box is reasonably busy, as it's the sole piece of hardware 
> running web, database, and mail operations for the Mars Society, an 
> international nonprofit group dedicated to space exploration. We 
> regularly send out newsletters to ~10,000 members, and our web site is 
> averaging ~50,000-100,000 hits/day.
> 
> The main portion of the web site is run via the Zope/Plone CMS system 
> (Plone 2.5, for anyone who may care). Recently, it's been slowing down 
> dramatically, and our Plone guy (not me -- I inherited the system and 
> can't stand it) can't figure out why. I've been diving into OS-related 
> issues, and in so doing, I ran across what appears to be a very high 
> number of context switches going on. Here's some sample output from 
> "vmstat 2":

A few hundred or thousand context switches per second is trivial load. 
That is not your problem.  Modern CPUs can do hundreds of thousands per 
second before it starts to become a problem.

Note that your system is 50% idle and spending almost no time in the 
kernel.  This basically means that only one core is doing work, which 
might be because you're not giving it enough work to do.  There are only 
1-2 running tasks for most of your trace, one of which is probably 
vmstat itself, so that means there is only one running server process 
(which can obviously only saturate at most 1 CPU).

The trace suggests that your performance problems are either in 
userland, or elsewhere in your network or application stack, possibly 
due to interactions between components.  Try to look at why the system 
is not being given enough work to keep it saturated.

Kris