[Bug 266873] www/apache24: creating forked processes at 100% CPU spinning in sched_yield()

From: <bugzilla-noreply_at_freebsd.org>
Date: Fri, 07 Oct 2022 00:26:51 UTC
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=266873

            Bug ID: 266873
           Summary: www/apache24: creating forked processes at 100% CPU
                    spinning in sched_yield()
           Product: Ports & Packages
           Version: Latest
          Hardware: amd64
                OS: Any
            Status: New
          Severity: Affects Some People
          Priority: ---
         Component: Individual Port(s)
          Assignee: apache@FreeBSD.org
          Reporter: edenist-fbz@edenist.net
             Flags: maintainer-feedback?(apache@FreeBSD.org)
          Assignee: apache@FreeBSD.org

Apache version: apache24-2.4.54 - build with poudriere
FreeBSD: 12.3-STABLE FreeBSD 12.3-STABLE stable/12-3a9e60169ee GENERIC amd64

Since updating OS and apache a couple of weeks ago, I have noticed that my
system is getting filled up with httpd processes consuming 100% CPU. 

Here is an example with one rogue process, but after a day of running the
system will be eventually full of httpd processes spinning at 100%. 

root   9947 100.0  0.0  207000  41380  -  RJ   10:59   18:42.98
/usr/local/sbin/httpd -DNOHTTPACCEPT
www   10243   1.4  0.1  310180 138488  -  SJ   11:17    0:01.44
/usr/local/sbin/httpd -DNOHTTPACCEPT
www   10059   0.4  0.1  242068  71156  -  SJ   11:06    0:03.63
/usr/local/sbin/httpd -DNOHTTPACCEPT
root   9738   0.0  0.0  207000  41388  -  SsJ  10:37    0:00.35
/usr/local/sbin/httpd -DNOHTTPACCEPT
www    9853   0.0  0.1  244096  70872  -  SJ   10:47    0:06.49
/usr/local/sbin/httpd -DNOHTTPACCEPT
www    9887   0.0  0.1  242500  69808  -  IJ   10:51    0:08.48
/usr/local/sbin/httpd -DNOHTTPACCEPT
www   10058   0.0  0.1  307876 136996  -  IJ   11:06    0:04.76
/usr/local/sbin/httpd -DNOHTTPACCEPT
www   10136   0.0  0.1  242592  71452  -  IJ   11:12    0:02.50
/usr/local/sbin/httpd -DNOHTTPACCEPT
www   10144   0.0  0.1  244644  71328  -  IJ   11:14    0:01.31
/usr/local/sbin/httpd -DNOHTTPACCEPT
www   10166   0.0  0.1  244648  73028  -  SJ   11:15    0:00.99
/usr/local/sbin/httpd -DNOHTTPACCEPT
www   10239   0.0  0.1  209800  61568  -  IJ   11:16    0:00.20
/usr/local/sbin/httpd -DNOHTTPACCEPT


Usual behavior with prefork will be an initial process running as root, which
then forks additional processes and sets them to run as www user. What appears
to be happening here is that when the fork occurs, something gets stuck before
it is able to switch to www. 

In the case of the rogue process above, output of truss -p 9947 is simply a
continuous loop of:
sched_yield()                                    = 0 (0x0)


For some extra context, I am running this inside a jail, hosting nextcloud with
mod_php80, mysql, redis. None of my other web apps are having this issue but it
could be something which only appears under load and nextcloud is certainly the
heaviest application here. 
Also, I tried switching over to php-fpm with mpm_event and the same issue was
encountered there, so it certainly feels like something core to apache's
forking code at play here.

-- 
You are receiving this mail because:
You are the assignee for the bug.