2 x quad-core system is slower that 2 x dual core on FreeBSD

[Alexey Popov]

Hi.

Ivan Voras wrote:
>> I have a large pool of web backends (Apache + mod_php5) with
>> 2 x Xeon 3.2GHz processors and 2 x Xeon 5120 dual-core processors. The
>> workload is mostly CPU-bound. I'm using 6-STABLE-amd64 and also tried
>> 7-STABLE. 
> If you haven't tried mod_fcgid, give it a try - it can dramatically
> benefit PHP applications. And with mod_fcgid, you can use apache with a
> multi-threaded MPM (i.e. worker-mpm).
We tried to run php + nginx via fastcgi interface without apache at all, 
but improvement was too little (~10% more request per second) to abandon 
the advantages of apache.

>> Now I'm trying to use new hardware with 2 x Xeon 5320 (quad-core), but
>> it can not work under the same load as dual-core. It shows up to 80%
>> system CPU load in top:
> On what version of FreeBSD is this? If it's 6-STABLE, this might be
> expected.
I have almost identical results on 6-STABLE and 7-STABLE. Maybe 7-STABLE 
performs a little better.

>> CPU states:  9.5% user,  0.0% nice, 79.9% system,  1.2% interrupt,  9.5%
>> idle
> Can you try hitting "S" to see if a kernel process is gobbling up CPU time?
There's no such a process:

last pid:  5266;  load averages: 24.67, 22.65, 17.44       up 0+03:56:38 
  17:09:37
121 processes: 41 running, 62 sleeping, 18 waiting
CPU states:  9.5% user,  0.0% nice, 82.0% system,  0.5% interrupt,  8.0% 
idle
Mem: 439M Active, 27M Inact, 80M Wired, 108K Cache, 58M Buf, 3341M Free
Swap: 2048M Total, 2048M Free

   PID USERNAME  PRI NICE   SIZE    RES STATE  C   TIME   WCPU COMMAND
  5090 www        -4    0 96572K 49464K RUN    5   2:59 23.39% httpd
  3748 www        -4    0 96172K 50060K RUN    4  14:21 23.19% httpd
  5092 www        -4    0 96412K 48060K RUN    4   2:57 23.19% httpd
  5095 www        -4    0 98148K 50688K RUN    5   2:57 22.75% httpd
  5088 www        -4    0 96664K 49120K RUN    4   3:02 22.56% httpd
  5098 www        -4    0 97404K 49864K RUN    3   2:57 22.56% httpd
  5106 www       118    0 97908K 49972K CPU7   6   2:57 22.51% httpd
  5084 www        -4    0 96012K 48164K RUN    5   3:01 22.46% httpd
  5081 www        -4    0 96636K 49700K RUN    0   3:01 22.36% httpd
  5109 www        -4    0 96844K 49188K RUN    3   2:51 22.36% httpd
  5108 www        -4    0 95808K 47508K RUN    5   3:00 22.31% httpd
  5085 www        -4    0 98244K 49560K RUN    4   2:58 21.88% httpd
  5104 www        -4    0 96836K 48956K CPU5   5   2:55 21.88% httpd
  5086 www       118    0 99140K 51264K CPU0   3   3:00 21.78% httpd
  5111 www        -4    0 96360K 48532K RUN    0   2:56 21.78% httpd
  5105 www        -4    0 96364K 47356K RUN    0   2:58 21.73% httpd
  5099 www        -4    0 94444K 47156K RUN    4   2:55 21.73% httpd
  5096 www        -4    0 96004K 48324K RUN    4   2:56 21.68% httpd
  5083 www       117    0 97712K 50344K RUN    2   3:03 21.63% httpd
  5094 www       118    0 97196K 49348K CPU3   6   2:56 21.58% httpd
  5103 www        -4    0 96040K 48808K RUN    4   2:58 21.48% httpd
  5089 www       118    0 96084K 47808K CPU2   4   2:59 21.34% httpd
  5082 www       117    0 96412K 48520K CPU6   5   3:00 21.29% httpd
  5107 www        -4    0 98172K 50332K RUN    4   2:55 21.29% httpd
  5091 www        -4    0 97460K 49504K RUN    0   2:56 20.95% httpd
  5100 www        -4    0 97188K 49400K RUN    4   2:56 20.65% httpd
  5110 www        -4    0 95168K 47436K RUN    5   2:59 20.56% httpd
  5087 www       116    0 98432K 51172K CPU4   5   2:55 20.31% httpd
  5097 www        -4    0 96428K 49124K RUN    4   2:59 20.21% httpd
  5102 www       117    0 96344K 48512K CPU3   4   3:01 19.82% httpd
  5093 www        -4    0 96512K 49948K RUN    4   2:55 19.82% httpd
  5101 www        -4    0 96012K 48968K RUN    3   3:01 19.48% httpd
    10 root      171   52     0K    16K RUN    7 174:56  7.86% idle: cpu7
    12 root      171   52     0K    16K RUN    5 174:44  7.86% idle: cpu5
    14 root      171   52     0K    16K RUN    3 175:04  7.62% idle: cpu3

>> Here's the output from 2xdual-core backend running under the same load
>> and with the same software:
> 
>> CPU states:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  0.0%
>> idle
> 
> This line is bogus - where is the load?
Sorry, probably it was my fault in copy&past.

last pid: 54690;  load averages:  3.47,  4.89,  5.18   up 42+02:07:51 
17:00:00
47 processes:  3 running, 43 sleeping, 1 zombie
CPU states: 56.0% user,  0.0% nice, 16.7% system,  1.6% interrupt, 25.7% 
idle
Mem: 2268M Active, 416M Inact, 277M Wired, 186M Cache, 214M Buf, 664M Free
Swap: 2048M Total, 1408K Used, 2047M Free

   PID USERNAME   THR PRI NICE   SIZE    RES STATE  C   TIME   WCPU COMMAND
54681 www          1 106    0 96916K 47792K CPU3   0   0:10 33.45% httpd
54652 www          1  20    0 97716K 48144K lockf  1   0:24 31.61% httpd
54680 www          1 106    0 96416K 46832K select 1   0:10 31.37% httpd
54686 www          1  20    0 97640K 45604K lockf  1   0:04 31.13% httpd
54651 www          1 104    0 96552K 46924K CPU1   1   0:25 29.50% httpd
54685 www          1 107    0 99124K 47300K select 3   0:03 25.93% httpd
54676 www          1 104    0 96492K 46368K select 1   0:09 25.14% httpd
54690 www          1  20    0 95816K 43584K lockf  1   0:01 23.81% httpd
54687 www          1  20    0 96360K 42892K lockf  1   0:02 22.48% httpd
54684 www          1 107    0 99120K 46152K select 3   0:02 13.18% httpd
   787 nobody       1   4    0  2079M  2077M kqread 1  20.7H  4.00% 
memcached
   660 root         1  96    0 18120K  2108K select 0  25:18  0.00% snmpd
   795 root         6  20    0 14652K  2992K kserel 1  13:28  0.00% 
bacula-fd

>> What can I do to make FreeBSD run faster on many-CPU systems???
> Except for trying 7-STABLE, there's not much you can do.
I've already tried it, but no luck.

With best regards,
Alxey Popov