2 x quad-core system is slower that 2 x dual core on FreeBSD

[Ivan Voras]

On 20/11/2007, Alexey Popov <lol at chistydom.ru> wrote:

> CPU states:  5.9% user,  0.0% nice, 81.3% system,  0.0% interrupt, 12.8% idle
> CPU states: 82.2% user,  0.0% nice, 13.8% system,  0.0% interrupt,  4.0% idle

Interesting coincidence: 1 CPU generates almost 8x less "sys time" then 8 CPUs.

But it seems that you have found something real. Inspired by your
problem I've done a simple measurement ("ab") on a 4-CPU (2x2 core
Opterons 2216 HE, PAE) machine I maintain, under these circumstances:

- a "heavy" PHP application
- FastCGI
- in this case, load of 4 clients
- on 6-STABLE

and I'm reporting similar findings:

last pid:  2254;  load averages:  1.43,  0.92,  0.69   up 71+08:23:06  18:00:31
153 processes: 8 running, 144 sleeping, 1 zombie
CPU states: 38.8% user,  0.0% nice, 48.4% system,  3.2% interrupt,  9.6% idle
Mem: 2321M Active, 1135M Inact, 313M Wired, 139M Cache, 112M Buf, 93M Free
Swap: 4500M Total, 336K Used, 4500M Free

  PID USERNAME  THR PRI NICE   SIZE    RES STATE  C   TIME   WCPU COMMAND
 2208 www         1  99    0   115M 19808K RUN    1   0:06 36.83% php-cgi
 2207 www         1 100    0   114M 19348K RUN    3   0:05 32.66% php-cgi
 1715 www         1  99    0   115M 23672K CPU0   0   0:24 27.83% php-cgi
 1710 www         1 101    0   114M 23460K RUN    1   0:31 22.17% php-cgi
 1882 www         1  99    0   115M 23392K CPU2   3   0:18 21.34% php-cgi
 1718 www         1   4    0   114M 22556K sbwait 0   0:21 19.14% php-cgi
 2677 pgsql       1   4    0   977M 55768K sbwait 0   0:00 28.00% postgres

We are not so performance bound as you so I didn't do measurements
earlier. I cannot "play" with settings on this machine as it is in
production, but ~~50% sys time (the measurement changes around 45% +/-
10%) seems too much.

On another 4-CPU machine (2x2 Xeons 5110, AMD64) with the same
application and benchmark setup, but RELENG_7, which is not yet in
production, the results are slightly different:

last pid: 66564;  load averages:  1.87,  0.48,  0.18   up 15+05:27:03  17:09:09
113 processes: 9 running, 104 sleeping
CPU states: 49.0% user,  0.0% nice, 28.8% system,  0.0% interrupt, 22.1% idle
Mem: 555M Active, 295M Inact, 884M Wired, 98M Cache, 213M Buf, 135M Free
Swap: 2047M Total, 2047M Free

  PID USERNAME   THR PRI NICE   SIZE    RES STATE  C   TIME   WCPU COMMAND
66557 www          1 109    0   105M 25340K RUN    3   0:14 64.99% php-cgi
66559 www          1 109    0   105M 25308K RUN    2   0:14 62.99% php-cgi
66561 www          1  98    0   105M 22196K RUN    0   0:01 12.99% php-cgi
66562 www          1  98    0   105M 22196K RUN    1   0:01 11.96% php-cgi
59043 nobody       1  47    0  7012K  3744K select 2   0:27  5.96% sqlcached
  774 pgsql        1  44    0   437M   112M select 2   3:55  0.00% postgres