Too Much Context Switching?

[alex at schnarff.com]

Quoting Kris Kennaway <kris at FreeBSD.org>:

> Michel Talon wrote:
>> On Mon, Jun 30, 2008 at 07:53:00PM +0200, Kris Kennaway wrote:
>>> Yep, it could be that -- what confuses me though is that it is 
>>> claimed that performance suddenly regressed.  If so then this 
>>> cannot be the underlying cause.
>>
>> It may be that the load has augmented to the point that contention
>> imposes a rapid regression on throughput.
>
> Yes, it could be that.  I don't know off-hand whether multiple threads
> are counted separately by vmstat (at a guess I'd say no), but ps/top/etc
> should show how many are active in the python process.

Just ran ktrace, and a bit of Googling seems to confirm my initial 
suspicion that the results I'm seeing are abnormal. The first several 
screenfulls of output look like this:

  52929 python2.4 1214867016.469416 CALL  kse_wakeup(0x811740c)
  52929 python2.4 0.000060 RET   kse_wakeup 0
  52929 python2.4 0.000008 RET   kse_release 0
  52929 python2.4 0.000040 CALL  kse_release(0x811df4c)
  52929 python2.4 0.000515 CALL  kse_wakeup(0x811740c)
  52929 python2.4 0.000012 RET   kse_wakeup 0
  52929 python2.4 0.000004 RET   kse_release 0
  52929 python2.4 0.000012 CALL  kse_release(0x811df4c)
  52929 python2.4 0.000365 CALL  kse_wakeup(0x811740c)
  52929 python2.4 0.000012 RET   kse_wakeup 0
  52929 python2.4 0.000003 RET   kse_release 0
  52929 python2.4 0.000010 CALL  kse_release(0x811df4c)
  52929 python2.4 0.000413 CALL  kse_wakeup(0x811740c)
  52929 python2.4 0.000011 RET   kse_wakeup 0
  52929 python2.4 0.000004 RET   kse_release 0
  52929 python2.4 0.000009 CALL  kse_release(0x811df4c)
  52929 python2.4 0.000393 CALL  kse_wakeup(0x811740c)
  52929 python2.4 0.000012 RET   kse_wakeup 0
  52929 python2.4 0.000004 RET   kse_release 0
  52929 python2.4 0.000009 CALL  kse_release(0x811df4c)

I may be mistaken, but it seems like that's a lot of unnecessary 
activity managing the threads; the confirmation I found came from 
http://arkiv.freebsd.se/?ml=freebsd-threads&a=2007-02&t=3178634.

Am I correct that this is abnormal behavior? If so, any idea what I may 
need to do to fix the issue?

Alex