2 x quad-core system is slower that 2 x dual core on FreeBSD
Alexey Vlasov
renton at df.ru
Sat Dec 1 13:58:00 PST 2007
Hi,
It seems that I'm not the only one who faced the problem that FreeBSD is
non productive on multiprocessors platforms.
I use OS Linux on my hosting for web-servers, base for all servers is
the same m/b S5000PAL ( SR1500), 2 quad kernel cpu Xeon E5320 or E5345,
8Gb RAM. I decided to install FreeBSD 6.2 i386 on one of the servers,
and the result was totally non productive.
Used software, Apache 2.2.6 (worker) as frontend Proxy, backend Apache
2.2.3 (prefork)
By the time we understood,that something was wrong with FreeBSD, there
had already been placed about 10 sites with high-capacity and about
hundred of usual ones. And this was the limit for FreeBSD. It came along
with a great amount of Context Switches, about hundred thousands.
I attached the log what was then happening with FreeBSD.
After playing with ab (ApacheBenchmark) options, it turned out that even
with the following options you can totally down the server:
./ab -n 100 -c 20 -t 30 http://somesite-freebsd.com
I copied at the same time somesite.com (php scripts) to Linux server,
launched ab with the same options, and saw that it has no influence on
work of the server. (And by the way there work about 1.5 virtual hosts
on that server)
All options for Apache on Linux and FreeBSD are the same:
FreeBSD:
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/
Benchmarking somesite-freebsd.com (be patient)
Finished 29 requests
Server Software: Apache/2.2.3
Server Hostname: somesite-freebsd.com
Server Port: 80
Document Path: /ab/
Document Length: 41450 bytes
Concurrency Level: 20
Time taken for tests: 30.44765 seconds
Complete requests: 29
Failed requests: 22
(Connect: 0, Length: 22, Exceptions: 0)
Write errors: 0
Total transferred: 1529557 bytes
HTML transferred: 1513497 bytes
Requests per second: 0.97 [#/sec] (mean)
Time per request: 20720.527 [ms] (mean)
Time per request: 1036.026 [ms] (mean, across all concurrent
requests)
Transfer rate: 49.69 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1760 1503.4 3002 3002
Processing: 866 13328 9460.5 13853 26246
Waiting: 139 2286 2319.0 1129 6764
Total: 871 15089 10642.6 16855 29248
Percentage of the requests served within a certain time (ms)
50% 16705
66% 22670
75% 25439
80% 26342
90% 29160
95% 29188
98% 29248
99% 29248
100% 29248 (longest request)
Linux: (the same site)
This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/
Benchmarking linux.1gb.ru (be patient)
Finished 814 requests
Server Software: Apache/2.2.3
Server Hostname: somesite-linux.com
Server Port: 80
Document Path: /ab/
Document Length: 41451 bytes
Concurrency Level: 20
Time taken for tests: 30.3216 seconds
Complete requests: 814
Failed requests: 759
(Connect: 0, Length: 759, Exceptions: 0)
Write errors: 0
Non-2xx responses: 1
Total transferred: 34430291 bytes
HTML transferred: 34126461 bytes
Requests per second: 27.13 [#/sec] (mean)
Time per request: 737.180 [ms] (mean)
Time per request: 36.859 [ms] (mean, across all concurrent
requests)
Transfer rate: 1120.65 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 214 725 575.2 557 5001
Waiting: 41 265 376.0 131 3280
Total: 214 725 575.2 557 5001
Percentage of the requests served within a certain time (ms)
50% 557
66% 716
75% 863
80% 967
90% 1398
95% 1749
98% 2529
99% 3064
100% 5001 (longest request)
# cat /etc/sysctl.conf
security.bsd.see_other_uids=0
kern.maxfiles=204800
kern.maxfilesperproc=202400
kernel:
machine i386
cpu I686_CPU
ident F1RNT1
options PAE
options SMP
options SCHED_4BSD
options PREEMPTION
options INET
options FFS
options SOFTUPDATES
options UFS_ACL
options UFS_DIRHASH
options NULLFS
options MD_ROOT
options CD9660
options PROCFS
options PSEUDOFS
options GEOM_GPT
options GEOM_LABEL
options GEOM_MIRROR
options COMPAT_43
options COMPAT_FREEBSD4
options COMPAT_FREEBSD5
options SCSI_DELAY=5000
options KTRACE
options SYSVSHM
options SYSVMSG
options SYSVSEM
options _KPOSIX_PRIORITY_SCHEDULING
options KBD_INSTALL_CDEV
options ADAPTIVE_GIANT
options LIBICONV
options NETSMB
options NETSMBCRYPTO
options LIBMCHAIN
options SMBFS
options IPFIREWALL
options IPFIREWALL_VERBOSE
options IPFIREWALL_VERBOSE_LIMIT=1000
options IPFIREWALL_DEFAULT_TO_ACCEPT
options IPFIREWALL_FORWARD
options IPDIVERT
options IPSTEALTH
options TCPDEBUG
options DUMMYNET
options IPSEC
options IPSEC_DEBUG
options ACCEPT_FILTER_DATA
options ACCEPT_FILTER_HTTP
# cat /etc/make.conf
CPUTYPE?=nocona
CFLAGS=-O2 -pipe
CXXFLAGS+=-fconserve-space
COPTFLAGS=-O2 -pipe
NO_CPU_CFLAGS=false
NO_CPU_COPTFLAGS=false
NO_MODULES=
--
BRGDS. Alexey Vlasov.
-------------- next part --------------
last pid: 31039; load averages: 51.33, 26.31, 18.09
up 15+22:58:54 16:47:30
692 processes: 80 running, 608 sleeping, 2 stopped, 1 zombie, 1 lock
CPU states: 2.4% user, 0.0% nice, 89.1% system, 1.6% interrupt, 6.9%
idle
Mem: 1587M Active, 5614M Inact, 416M Wired, 369M Cache, 112M Buf, 14M
Free
Swap: 16G Total, 46M Used, 16G Free
PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND
31027 w_istok2_d424 1 -4 0 76240K 36532K ufs 4 0:00 9.74% httpd
30826 w_nova-i_7638 1 -4 0 79340K 40116K ufs 4 0:02 7.78% httpd
31025 w_shop2y_b290 1 -4 0 76212K 36512K RUN 7 0:00 7.70% httpd
28349 w_urtest_7dc3 1 -4 0 79616K 39496K RUN 4 1:27 7.32% httpd
30801 w_new-vw_5e17 1 -4 0 76092K 37184K ufs 5 0:02 7.25% httpd
30914 w_avtolo_3761 1 -4 0 76316K 37304K RUN 7 0:01 7.18% httpd
29952 w_svekon_9de8 1 -4 0 77976K 42916K RUN 6 0:11 7.04% httpd
29875 w_lancer_12cf 1 -4 0 79584K 41408K RUN 6 0:12 6.94% httpd
29818 w_iprit-_be41 1 -4 0 77340K 39880K RUN 6 0:13 6.92% httpd
30811 w_dfase__508a 1 -4 0 76456K 37928K RUN 4 0:02 6.91% httpd
30200 w_iprit-_be41 1 -4 0 76292K 37260K ufs 1 0:07 6.86% httpd
30445 w_new-vw_5e17 1 119 0 77048K 40196K CPU3 0 0:04 6.74% httpd
30025 w_forum-_89b3 1 -4 0 76824K 41240K RUN 4 0:10 6.71% httpd
30687 w_new-vw_5e17 1 -4 0 76232K 38616K RUN 1 0:02 6.69% httpd
30561 w_avtolo_3761 1 105 0 76512K 39500K select 2 0:03 6.67% httpd
30519 w_new-vw_5e17 1 -4 0 76784K 39396K ufs 4 0:04 6.62% httpd
31018 w_tucson_bd87 1 -4 0 76000K 36464K RUN 6 0:00 6.62% httpd
30867 w_as-vid_ac7e 1 -4 0 76356K 37452K RUN 6 0:01 6.60% httpd
30997 w_rfscla_06fc 1 -4 0 76820K 38652K RUN 2 0:01 6.60% httpd
31015 w_tucson_bd87 1 -4 0 76024K 36908K RUN 1 0:00 6.59% httpd
30532 w_shop2y_b290 1 -4 0 76904K 38232K ufs 4 0:04 6.55% httpd
30936 w_tucson_bd87 1 -4 0 77836K 39740K RUN 6 0:01 6.53% httpd
30376 w_vannay_4606 1 -4 0 78252K 41596K RUN 1 0:05 6.53% httpd
30452 w_new-vw_5e17 1 -4 0 77356K 40124K RUN 6 0:04 6.50% httpd
30201 w_berega_3091 1 -4 0 81636K 42992K RUN 4 0:07 6.46% httpd
30840 w_forum-_89b3 1 -4 0 76412K 38752K RUN 4 0:02 6.44% httpd
30809 w_rm-for_636d 1 -4 0 76148K 38028K RUN 6 0:02 6.43% httpd
30700 w_ott-gu_2def 1 -4 0 76888K 39272K RUN 4 0:02 6.40% httpd
29971 w_mobi-m_2e46 1 -4 0 76872K 41484K RUN 4 0:05 6.40% httpd
31016 w_wplane_ebed 1 -4 0 76320K 37056K RUN 7 0:00 6.39% httpd
30839 w_lyubar_f1e2 1 -4 0 77332K 39788K RUN 4 0:02 6.36% httpd
31004 w_digitr_77d4 1 -4 0 76016K 36656K CPU5 5 0:01 6.36% httpd
# vmstat
procs memory page disks faults cpu
r b w avm fre flt re pi po fr sr ad4 ad6 in sy cs us sy id
110 9 0 10884836 384132 1167 2 2 1 1340 273 0 0 19 809 1282 13 9 78
110 10 0 10950600 382964 4530 0 0 0 3968 0 1 13 3925 35129 162137 9 87 4
131 9 0 10855252 387356 3552 0 2 0 4782 0 1 9 2505 27529 166233 5 92 3
129 11 0 10924792 379056 6723 0 4 0 4658 0 1 26 4015 29908 162047 3 94 3
122 10 1 10936140 367864 5289 0 2 0 2145 0 0 17 3788 27790 162641 3 95 3
66 55 0 10807504 378600 4236 0 4 0 6818 0 1 6 3164 26302 167760 2 95 2
128 11 0 10815488 374308 4092 0 3 0 2910 0 1 13 2962 32777 167559 2 93 5
80 42 1 10784792 377988 4592 0 3 0 5821 0 5 13 3296 28117 163950 3 94 2
64 62 0 10785528 377076 5640 0 3 0 5238 0 1 6 2890 31425 163924 3 94 3
52 77 0 10656100 386120 2994 0 1 0 4768 0 1 8 2427 23684 159985 2 95 3
70 52 0 10591528 393632 3381 0 0 0 5335 0 1 4 3010 27593 158225 3 94 3
105 9 0 10549508 402488 5426 0 0 0 7436 0 2 7 2318 31753 168856 3 94 4
77 40 0 10554208 397512 5848 0 0 0 4448 0 1 9 2275 31423 156746 3 92 5
114 9 4 10483152 403624 3369 0 0 0 4974 0 0 5 2243 27000 160225 2 93 5
109 9 0 10550332 394796 4972 0 1 0 2708 0 1 11 2515 26056 158628 3 92 5
125 9 0 10518288 423240 4256 0 3 0 11099 0 1 9 2779 28773 158967 3 93 4
118 11 0 10529144 415176 5887 4 3 0 3670 0 1 11 2595 30282 159381 2 95 3
116 9 0 10497144 418012 3407 0 1 0 3929 0 1 11 2815 24890 159056 2 93 5
119 9 0 10463764 419260 3391 0 0 0 3797 0 1 9 2558 28508 159086 2 94 3
5 116 0 10436304 420156 4263 0 0 0 4368 0 24 8 3348 27563 159025 3 94 4
84 43 0 10563104 412824 5012 0 0 0 3630 0 2 32 2658 32205 158341 1 94 5
82 59 0 10620056 415188 6158 4 2 0 4858 0 11 11 2743 30929 159186 3 94 4
87 39 0 10453904 431328 3867 0 0 0 8894 0 1 5 2560 28903 168316 2 95 3
102 37 2 10642596 417724 6146 0 1 0 3285 0 5 11 2396 35490 158978 2 95 4
145 10 0 10470216 461180 5202 4 0 0 15462 0 77 10 2522 30525 163784 4 93 3
122 12 0 10507604 452820 4561 0 0 50 2341 0 138 6 2368 31650 158428 3 94 3
108 29 0 10669764 439152 4808 0 0 0 1294 0 4 8 3456 27089 166675 2 95 3
124 10 0 10631948 444132 6171 0 0 0 7239 0 6 6 2875 31822 167193 4 94 3
7 users Load 54.35 28.49 23.64 Nov 27 17:04
Mem:KB REAL VIRTUAL VN PAGER SWAP
PAGER
Tot Share Tot Share Free in out in out
Act 927804 34556 8233412 53740 489892 count
All 8071840 267008983194432 3143860 pages
Interrupts
Proc:r p d s w Csw Trp Sys Int Sof Flt 1189 cow 16584
total
89 9454 153k 533325086 2022 309 5066 425580 wire 1:
atkb
1499680 act
14: ata
90.0%Sys 0.6%Intr 2.9%User 0.0%Nice 6.4%Idl 5776020 inact 546
18: em0
| | | | | | | | | | 425236 cache 31
20: ata
=============================================>> 64968 free 2001
cpu0: time
daefr 2002
cpu1: time
Namei Name-cache Dir-cache 2333 prcfr 2001
cpu2: time
Calls hits % hits % react 2001
cpu3: time
207699 207669 100 5 0 pdwak 2000
cpu7: time
3514 zfod pdpgs 2001
cpu4: time
Disks ad4 ad6 ad8 3463 ozfod intrn 2000
cpu6: time
KB/t 16.00 4.20 6.90 98 %slo-z 114784 buf 2001
cpu5: time
tps 13 1 1 3077 tfree 229 dirtybuf
MB/s 0.21 0.00 0.01 100000 desiredvnodes
% busy 0 1 1 89091 numvnodes
24971 freevnodes
More information about the freebsd-stable
mailing list