Anyone interested in improving postgresql scaling?

[Kevin Way]

Robert Watson wrote:
>
> On Tue, 10 Apr 2007, Kevin Way wrote:
>
>> Kris Kennaway wrote:
>>> If so, then your task is the following:
>>>
>>> Make SYSV semaphores less dumb about process wakeups.  Currently 
>>> whenever the semaphore state changes, all processes sleeping on the 
>>> semaphore are woken, even if we only have released enough resources 
>>> for one waiting process to claim.  i.e. there is a thundering herd 
>>> wakeup situation which destroys performance at high loads.  Fixing 
>>> this will involve replacing the wakeup() calls with appropriate 
>>> amounts of wakeup_one().
>> Could this cause problem cause a situation where an 8-Core system was 
>> 50-75% slower than an otherwise equivalent 2-Core system?
>>
>> I have a graph of my sysbench/pgsql results here:
>>
>>   http://blog.insidesystems.net/files/sysctl-pgsq-amd64-wtf.png
>>
>> As the graph shows, the 8-core system is about half the speed of the 
>> 2-core system at 2 simultaneous threads, and it decays down to 
>> approximately 1/4 the speed of the 2-core system as the # of threads 
>> hits 5.
>>
>> All other (non-pgsql, non-sysv) tests came back approximately as 
>> expected, but I'm left wondering if I did something wrong, or if 8 
>> cpus are slower than 2, when it comes to Postgres on currently 
>> available FreeBSD.
>>
>> Kevin Way Inside Systems, Inc.
>>
>> (Full detail of what I did is available at: 
>> http://blog.insidesystems.net/articles/2007/04/09/what-did-i-do-wrong )
>
> Would you be able to re-run these tests trying a recent 7.x kernel?  
> If so, make sure you build a non-debugging kernel, and try two 
> variants: one with SCHED_4BSD, and one with SCHED_ULE.
>
> Another experiment that would be interesting is to try using 
> device.hints to disable various numbers of CPUs on your 8-core system, 
> and see how the performance graph changes as the number of enabled 
> CPUs changes.
>
> Robert N M Watson
> Computer Laboratory
> University of Cambridge
I built 7.0 as of 6 days ago, and ran the same test using 8-cores, ULE 
and 4BSD.  The results are available at:

http://blog.insidesystems.net/articles/2007/04/11/postgresql-scaling-on-6-2-and-7-0

Unfortunately, I can't run the additional tests, as the machines in 
question have now been deployed to production.  I should have similar 
equipment available in a few weeks, and will try the other tests at that 
point.

Kevin Way
Inside Systems, Inc.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 4478 bytes
Desc: S/MIME Cryptographic Signature
Url : http://lists.freebsd.org/pipermail/freebsd-performance/attachments/20070417/32f5d9fc/smime.bin