php-cgi frozen with sbwait when SMP enable

Ken Chen ken73.chen at gmail.com
Tue Nov 11 06:36:38 PST 2008


The report from lighttpd looks fine:

cache.cached-itmes: 98293
cache.hitrate(%): 96
cache.memory-inuse(KB): 6143
fastcgi.active-requests: 16
fastcgi.backend.0.0.connected: 419008
fastcgi.backend.0.0.died: 0
fastcgi.backend.0.0.disabled: 0
fastcgi.backend.0.0.load: 16
fastcgi.backend.0.0.overloaded: 0
fastcgi.backend.0.load: 17
fastcgi.requests: 419008

But at this moment:

web4# ps alx | grep php-cgi | grep -v grep | awk '{print $9}' | sort |
uniq -c | sort -n
   1 biord
   1 wait
   2 -
  16 sbwait
  29 accept

web4# ps alx | grep php-cgi | grep -v grep | grep sbwait
65534 61392 61384   1   4 -15 182328 69312 sbwait I<    ??    0:53.39
/usr/local/bin/php-cgi
65534 61399 61384   0   4 -15 182328 71112 sbwait I<    ??    1:09.60
/usr/local/bin/php-cgi
65534 61409 61384   0   4 -15 182328 72812 sbwait I<    ??    1:39.40
/usr/local/bin/php-cgi
65534 61411 61384   0   4 -15 183352 74536 sbwait I<    ??    1:49.08
/usr/local/bin/php-cgi
65534 61412 61384   0   4 -15 183352 74508 sbwait I<    ??    1:33.31
/usr/local/bin/php-cgi
65534 61414 61384   0   4 -15 182328 62860 sbwait I<    ??    0:28.81
/usr/local/bin/php-cgi
65534 61418 61384   0   4 -15 182328 71448 sbwait I<    ??    1:17.56
/usr/local/bin/php-cgi
65534 61426 61384   0   4 -15 183352 60456 sbwait I<    ??    0:22.16
/usr/local/bin/php-cgi
65534 71529 61384   0   4 -15 182328 74144 sbwait I<    ??    0:36.87
/usr/local/bin/php-cgi
65534 71555 61384   0   4 -15 182328 72820 sbwait I<    ??    0:19.19
/usr/local/bin/php-cgi
65534 71556 61384   0   4 -15 182328 74452 sbwait I<    ??    0:38.27
/usr/local/bin/php-cgi
65534 71590 61384   0   4 -15 182328 76828 sbwait I<    ??    0:57.42
/usr/local/bin/php-cgi
65534 71594 61384   0   4 -15 182328 75576 sbwait I<    ??    0:46.50
/usr/local/bin/php-cgi
65534 71595 61384   0   4 -15 182328 84048 sbwait I<    ??    1:52.15
/usr/local/bin/php-cgi
65534 77285 61384   0   4 -15 182328 88280 sbwait S<    ??    0:15.22
/usr/local/bin/php-cgi
65534 77288 61384   3   4 -15 182328 88808 sbwait S<    ??    0:14.43
/usr/local/bin/php-cgi
65534 77317 61384   0   4 -15 182328 88912 sbwait S<    ??    0:12.79
/usr/local/bin/php-cgi
65534 77323 61384   0   4 -15 182328 89140 sbwait S<    ??    0:13.51
/usr/local/bin/php-cgi
65534 77359 61384   6   4 -15 182328 88200 sbwait S<    ??    0:13.04
/usr/local/bin/php-cgi
65534 77372 61384   2   4 -15 182328 89200 sbwait S<    ??    0:12.16
/usr/local/bin/php-cgi
65534 77392 61384   1   4 -15 181304 87200 sbwait S<    ??    0:11.02
/usr/local/bin/php-cgi
65534 77401 61384   1   4 -15 182328 88800 sbwait S<    ??    0:12.49
/usr/local/bin/php-cgi

The PID of php-cgi which are less than 71595 are frozen.



2008/11/10 Anton - Valqk <lists at lozenetz.org>

> You can try taking look to lighttpd status and fcgi processes status
> like this:
> server.modules   += ( "mod_status" )
> status.status-url = "/server-status"
> status.statistics-url = "/sstatus1"
>
> status.statistics-url gives info for each fastcgi like this:
>
> fastcgi.active-requests: 0
> fastcgi.backend.backend1.0.connected: 12493970
> fastcgi.backend.backend1.0.died: 0
> fastcgi.backend.backend1.0.disabled: 0
> fastcgi.backend.backend1.0.load: 0
> fastcgi.backend.backend1.0.overloaded: 0
> fastcgi.backend.backend1.load: 1
> fastcgi.requests: 19479062
>
>
> etc... read what each means on lighttpd site...
> pls tell what caused this, it'd be very interesting to me!
>
> cheers,
> valqk.
>
> Ken Chen wrote:
> > I capture something.
> >
> > Please check the PID 57776. It's CPU time never change since my previous
> > mail here.
> >
> > web4# ps alx | grep php-cgi | grep -v grep | grep sbwait
> > 65534 57776 47240   0   4  0 182328 84984 sbwait I     ??    2:02.12
> > /usr/local/bin/php-cgi
> > 65534 57801 47240   0   4  0 182328 82408 sbwait I     ??    0:19.97
> > /usr/local/bin/php-cgi
> > 65534 57809 47240   0   4  0 182328 84096 sbwait I     ??    1:12.03
> > /usr/local/bin/php-cgi
> > 65534 57823 47240   0   4  0 182328 84492 sbwait I     ??    2:04.21
> > /usr/local/bin/php-cgi
> > 65534 57833 47240   0   4  0 183352 83316 sbwait I     ??    0:28.62
> > /usr/local/bin/php-cgi
> > 65534 57866 47240   0   4  0 182328 79952 sbwait I     ??    0:05.92
> > /usr/local/bin/php-cgi
> > 65534 57870 47240   0   4  0 182328 83184 sbwait I     ??    0:56.83
> > /usr/local/bin/php-cgi
> > 65534 57871 47240   0   4  0 182328 83388 sbwait I     ??    0:54.96
> > /usr/local/bin/php-cgi
> > 65534 57891 47240   0   4  0 182328 84436 sbwait I     ??    1:58.32
> > /usr/local/bin/php-cgi
> > 65534 57925 47240   0   4  0 182328 84380 sbwait I     ??    2:03.53
> > /usr/local/bin/php-cgi
> > 65534 65944 47240   0   4  0 182328 84184 sbwait I     ??    0:39.97
> > /usr/local/bin/php-cgi
> > 65534 65952 47240   0   4  0 182328 84408 sbwait I     ??    0:21.37
> > /usr/local/bin/php-cgi
> > 65534 66007 47240   0   4  0 183352 90960 sbwait I     ??    1:16.81
> > /usr/local/bin/php-cgi
> > 65534 66014 47240   5   4  0 182328 92748 sbwait S     ??    1:41.23
> > /usr/local/bin/php-cgi
> > 65534 66038 47240   1   4  0 182328 91900 sbwait I     ??    1:38.04
> > /usr/local/bin/php-cgi
> > 65534 66060 47240   0   4  0 182328 90048 sbwait I     ??    1:15.46
> > /usr/local/bin/php-cgi
> > 65534 66078 47240   3   4  0 182328 92224 sbwait S     ??    1:39.66
> > /usr/local/bin/php-cgi
> > web4# top -b
> > last pid: 70768;  load averages:  1.62,  1.65,  1.43  up 4+15:56:06
> > 22:53:48
> > 85 processes:  1 running, 84 sleeping
> >
> > Mem: 492M Active, 1204M Inact, 218M Wired, 60M Cache, 112M Buf, 27M Free
> > Swap: 2019M Total, 20K Used, 2019M Free
> >
> >
> >   PID USERNAME  THR PRI NICE   SIZE    RES STATE  C   TIME   WCPU COMMAND
> > 69544 nobody      1   8    0   203M 38500K nanslp 1   6:31 11.33% php
> > 47290 nobody      1   4    0   101M    98M kqread 1  30:42  2.98%
> lighttpd
> > 66526 nobody      1   4    0   178M 92796K accept 1   1:40  1.12% php-cgi
> > 66077 nobody      1   4    0   178M 92512K accept 0   1:49  1.07% php-cgi
> > 65921 nobody      1   4    0   178M 92696K accept 0   1:43  0.98% php-cgi
> > 65968 nobody      1   4    0   178M 92484K accept 0   1:43  0.93% php-cgi
> > 66017 nobody      1   4    0   178M 92444K accept 0   1:50  0.88% php-cgi
> > 65979 nobody      1   4    0   178M 92676K accept 1   1:44  0.88% php-cgi
> > 66424 nobody      1   4    0   178M 92928K accept 1   1:36  0.88% php-cgi
> > 65938 nobody      1   4    0   178M 92336K accept 1   1:52  0.73% php-cgi
> > 65951 nobody      1   4    0   178M 92704K accept 0   1:48  0.73% php-cgi
> > 66016 nobody      1   4    0   178M 92232K accept 1   1:41  0.73% php-cgi
> > 65950 nobody      1   4    0   178M 93192K accept 0   1:51  0.68% php-cgi
> > 65999 nobody      1   4    0   178M 92940K accept 1   1:46  0.63% php-cgi
> > 66008 nobody      1   4    0   178M 93000K accept 1   1:46  0.63% php-cgi
> > 69286 nobody      1   4    0   178M 92208K accept 1   0:37  0.63% php-cgi
> > 47289 nobody      1   4    0 73400K 70640K kqread 1  12:02  0.59%
> lighttpd
> > 65980 nobody      1   4    0   178M 93156K accept 1   1:51  0.59% php-cgi
> > 2008/11/7 Ivan Voras <ivoras at freebsd.org>
> >
> >
> >> Ken Chen wrote:
> >>
> >>> Oh.. sorry, I forgot to provide the information of my environment.
> >>>
> >>> web4# php-cgi -v
> >>> PHP 5.2.6 (cgi-fcgi) (built: Nov  2 2008 11:16:30)
> >>> Copyright (c) 1997-2008 The PHP Group
> >>> Zend Engine v2.2.0, Copyright (c) 1998-2008 Zend Technologies
> >>>     with XCache v1.2.2, Copyright (c) 2005-2007, by mOo
> >>> web4# /usr/local/lighttpd/sbin/lighttpd -v
> >>> lighttpd-1.4.19 - a light and fast webserver
> >>> Build-Date: Sep  1 2008 16:58:51
> >>> web4# uname -a
> >>> FreeBSD web4.xxxx.com 7.0-RELEASE-p5 FreeBSD 7.0-RELEASE-p5 #11: Mon
> Nov
> >>>
> >>  3
> >>
> >>> 01:10:36 CST 2008     root at web4.xxxx.com:/usr/obj/usr/src/sys/WEB4
>  i386
> >>> web4# ps alx | grep php-cgi | grep -v grep | grep sbwait
> >>> 65534 57776 47240   0   4  0 182328 84984 sbwait I     ??    2:02.12
> >>> /usr/local/bin/php-cgi
> >>> 65534 57801 47240   0   4  0 182328 82408 sbwait I     ??    0:19.97
> >>> /usr/local/bin/php-cgi
> >>> 65534 57809 47240   0   4  0 182328 84096 sbwait I     ??    1:12.03
> >>> /usr/local/bin/php-cgi
> >>> 65534 57823 47240   0   4  0 182328 84492 sbwait I     ??    2:04.21
> >>> /usr/local/bin/php-cgi
> >>> 65534 57833 47240   0   4  0 183352 83316 sbwait I     ??    0:28.62
> >>> /usr/local/bin/php-cgi
> >>> 65534 57866 47240   0   4  0 182328 79952 sbwait I     ??    0:05.92
> >>> /usr/local/bin/php-cgi
> >>> 65534 57870 47240   0   4  0 182328 83184 sbwait I     ??    0:56.83
> >>> /usr/local/bin/php-cgi
> >>> 65534 57871 47240   0   4  0 182328 83388 sbwait I     ??    0:54.96
> >>> /usr/local/bin/php-cgi
> >>> 65534 57891 47240   0   4  0 182328 84436 sbwait I     ??    1:58.32
> >>> /usr/local/bin/php-cgi
> >>> 65534 57925 47240   0   4  0 182328 84380 sbwait I     ??    2:03.53
> >>> /usr/local/bin/php-cgi
> >>> 65534 65944 47240   0   4  0 182328 84184 sbwait I     ??    0:39.97
> >>> /usr/local/bin/php-cgi
> >>> 65534 65952 47240   0   4  0 182328 84408 sbwait I     ??    0:21.37
> >>> /usr/local/bin/php-cgi
> >>>
> >> This does seem a bit unusual, but seeing that your execution times are
> >> not null it might that the PHP servers are actually doing some useful
> >> work. You should have a mixture of various states in PHP - do they show
> >> up in top?
> >>
> >> My own example is:
> >>
> >> last pid: 77421;  load averages:  2.82,  2.59,  2.13
> >>                                                            up
> >> 55+16:58:49  15:48:16
> >> 209 processes: 2 running, 206 sleeping, 1 zombie
> >> CPU: 49.8% user,  0.0% nice,  2.8% system,  0.0% interrupt, 47.4% idle
> >> Mem: 1493M Active, 1583M Inact, 278M Wired, 139M Cache, 112M Buf, 505M
> Free
> >> Swap: 4500M Total, 416M Used, 4084M Free, 9% Inuse
> >>
> >>  PID USERNAME  THR PRI NICE   SIZE    RES STATE  C   TIME   WCPU COMMAND
> >> 75863 www         1   4    0   162M 50020K sbwait 3   2:54 36.77%
> php-cgi
> >> 76830 www         1 103    0   156M 41556K CPU2   3   1:28 36.77%
> php-cgi
> >> 76834 www         1   4    0   163M 56628K sbwait 0   2:23 33.59%
> php-cgi
> >> 76019 www         1   4    0   150M 38948K accept 3   3:12 20.56%
> php-cgi
> >> 76825 www         1   4    0   158M 42912K accept 2   1:21 18.16%
> php-cgi
> >> 76846 www         1   4    0   162M 42600K sbwait 1   1:07 14.36%
> php-cgi
> >> 76835 www         1   4    0   151M 39948K accept 2   1:28 12.60%
> php-cgi
> >> 76829 www         1   4    0   150M 36564K sbwait 2   1:46  2.98%
> php-cgi
> >>
> >> This is unusually high load, a spike, for this server but it has many
> >> cores and it's stable. It's also running 7.1-PRERELEASE.
> >>
> >>
> >>
> > _______________________________________________
> > freebsd-stable at freebsd.org mailing list
> > http://lists.freebsd.org/mailman/listinfo/freebsd-stable
> > To unsubscribe, send any mail to "freebsd-stable-unsubscribe at freebsd.org
> "
> >
> >
>
>


More information about the freebsd-stable mailing list