Too Much Context Switching?

alex at schnarff.com alex at schnarff.com
Mon Jun 30 19:21:44 UTC 2008


>>> In 6.x. the default thread library is quite inefficient although it 
>>> can make use of multiple CPUs (again, providing the application is 
>>> giving them work to do).  For multi-threaded performance you will 
>>> be better off switching to the libthr library (see libmap.conf(5)) 
>>> or updating to 7.0 (where it is the default).  This isn't likely to 
>>> be the underlying issue if you are trying to debug a loss of 
>>> performance relative to the same configuration in the past though.
>>
>> Indeed Plone is written in python, and python has a "Big Giant Lock"
>> inside which insures that only one thread can execute, in order to
>> protect the python structures. This lock is only released under special
>> circumstances, such as doing IO. Hence it is necessary to run several
>> instances of python programs and do synchronization work, if one wants
>> to make use of several CPUs, or use python threads, and immediately 
>> make some IOs, or similar techniques. It may be that using Jython, if
>> possible, yields better threading behavior. When doing some work
>> according to these ideas, i had found quite severe contention, and this
>> was not cured when switching native threading libraries (libksd, libthr,
>> etc.). The problem is really inside python.
>
> Yep, it could be that -- what confuses me though is that it is claimed
> that performance suddenly regressed.  If so then this cannot be the
> underlying cause.
>

It's actually been a long, slow, steady degradation of performance as 
best I can tell, that's recently just reached proportions that are so 
ridiculous that it's gone from "this sucks but I can deal" to "this is 
completely unusable." The system has been slow from the start, just not 
this slow. I guess I'll need to investigate this...and while I know 
that Python is somewhat off-topic, if anyone here has any suggestions 
on where to start, they'd be much appreciated. :-)

Alex


More information about the freebsd-questions mailing list