maintainer-feedback requested: [Bug 266873] www/apache24: creating forked processes at 100% CPU spinning in sched_yield()

Bugzilla Automation <bugzilla@FreeBSD.org> has asked freebsd-apache (Nobody)
<apache@FreeBSD.org> for maintainer-feedback:
Bug 266873: www/apache24: creating forked processes at 100% CPU spinning in
sched_yield()
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=266873



--- Description ---
Apache version: apache24-2.4.54 - build with poudriere
FreeBSD: 12.3-STABLE FreeBSD 12.3-STABLE stable/12-3a9e60169ee GENERIC amd64

Since updating OS and apache a couple of weeks ago, I have noticed that my
system is getting filled up with httpd processes consuming 100% CPU. 

Here is an example with one rogue process, but after a day of running the
system will be eventually full of httpd processes spinning at 100%. 

root   9947 100.0  0.0	207000	41380  -  RJ   10:59   18:42.98
/usr/local/sbin/httpd -DNOHTTPACCEPT
www   10243   1.4  0.1	310180 138488  -  SJ   11:17	0:01.44
/usr/local/sbin/httpd -DNOHTTPACCEPT
www   10059   0.4  0.1	242068	71156  -  SJ   11:06	0:03.63
/usr/local/sbin/httpd -DNOHTTPACCEPT
root   9738   0.0  0.0	207000	41388  -  SsJ  10:37	0:00.35
/usr/local/sbin/httpd -DNOHTTPACCEPT
www    9853   0.0  0.1	244096	70872  -  SJ   10:47	0:06.49
/usr/local/sbin/httpd -DNOHTTPACCEPT
www    9887   0.0  0.1	242500	69808  -  IJ   10:51	0:08.48
/usr/local/sbin/httpd -DNOHTTPACCEPT
www   10058   0.0  0.1	307876 136996  -  IJ   11:06	0:04.76
/usr/local/sbin/httpd -DNOHTTPACCEPT
www   10136   0.0  0.1	242592	71452  -  IJ   11:12	0:02.50
/usr/local/sbin/httpd -DNOHTTPACCEPT
www   10144   0.0  0.1	244644	71328  -  IJ   11:14	0:01.31
/usr/local/sbin/httpd -DNOHTTPACCEPT
www   10166   0.0  0.1	244648	73028  -  SJ   11:15	0:00.99
/usr/local/sbin/httpd -DNOHTTPACCEPT
www   10239   0.0  0.1	209800	61568  -  IJ   11:16	0:00.20
/usr/local/sbin/httpd -DNOHTTPACCEPT


Usual behavior with prefork will be an initial process running as root, which
then forks additional processes and sets them to run as www user. What appears
to be happening here is that when the fork occurs, something gets stuck before
it is able to switch to www. 

In the case of the rogue process above, output of truss -p 9947 is simply a
continuous loop of:
sched_yield()					 = 0 (0x0)


For some extra context, I am running this inside a jail, hosting nextcloud with
mod_php80, mysql, redis. None of my other web apps are having this issue but it
could be something which only appears under load and nextcloud is certainly the
heaviest application here. 
Also, I tried switching over to php-fpm with mpm_event and the same issue was
encountered there, so it certainly feels like something core to apache's
forking code at play here.