PostgreSQL performance scaling

[Ivan Voras]

This is not a request for help but a report, in case it helps developers 
or someone in the future. The setup is:

AMD64 machine, 24 GB RAM, 2x6-core Xeon CPU + HTT (24 logical CPUs)
FreeBSD 8.1-stable, AMD64
PostgreSQL 9.0.1, 10 GB shared buffers, using pgbench with a scale 
factor of 500 (7.5 GB database)

with pgbench -S (SELECT-queries only, no disk IO) the performance curve is:

-c#	result
4	33549
8	64864
12	79491
16	79887
20	66957
24	52576
28	50406
32	49491
40	45535
50	39499
75	29415

After 16 clients (which is still good since there are only 12 "real" 
cores in the system), the performance drops sharply, and looking at the 
processes' state, most of them seem to eat away system call (i.e. 
executing in the kernel) in states "semwait" and "sbwait", i.e. 
semaphore wait and socket buffer wait, for example:

  3107 pgsql       1  62    0 10533M   439M CPU1    0   0:02 13.57% postgres
  3105 pgsql       1  63    0 10533M   438M CPU9    9   0:02 13.57% postgres
  3109 pgsql       1  62    0 10533M   440M sbwait 13   0:02 13.48% postgres
  3106 pgsql       1  61    0 10533M   445M sbwait  8   0:02 13.48% postgres
  3118 pgsql       1  62    0 10533M   431M sbwait 21   0:02 13.48% postgres
  3114 pgsql       1  63    0 10533M   434M sbwait 19   0:02 13.38% postgres
  3122 pgsql       1  63    0 10533M   428M sbwait 15   0:02 13.28% postgres
  3108 pgsql       1  63    0 10533M   439M sbwait  5   0:02 13.18% postgres
  3116 pgsql       1  62    0 10533M   432M sbwait 11   0:02 13.18% postgres
  3113 pgsql       1  62    0 10533M   430M semwai 20   0:02 13.18% postgres
  3115 pgsql       1  62    0 10533M   428M RUN    14   0:02 13.18% postgres

The "semwait" part is from PostgreSQL - probably shared buffer locking, 
but there's a large number of processes regularly in sbwait - maybe 
something can be optimized here?

This is IPC over Unix sockets.