Runaway Apache

Mark Edwards mark at antsclimbtree.com
Sat Dec 18 13:25:01 PST 2004


In the last week or so, my FreeBSD 4.10p5 server has started locking up 
every day or so, to the point where it becomes unusable and must be 
rebooted to resume service.  I've noticed that when it happens, the 
following type of thing appears in /var/log/httpd-error.log

[Sat Dec 18 13:00:18 2004] [error] child process 248 still did not 
exit, sending a SIGKILL
[Sat Dec 18 13:00:18 2004] [error] child process 464 still did not 
exit, sending a SIGKILL
[Sat Dec 18 13:00:18 2004] [error] child process 465 still did not 
exit, sending a SIGKILL
[Sat Dec 18 13:00:18 2004] [error] child process 466 still did not 
exit, sending a SIGKILL
[Sat Dec 18 13:00:18 2004] [error] child process 554 still did not 
exit, sending a SIGKILL
[Sat Dec 18 13:00:18 2004] [error] child process 2121 still did not 
exit, sending a SIGKILL
[Sat Dec 18 13:00:18 2004] [error] child process 2126 still did not 
exit, sending a SIGKILL
[Sat Dec 18 13:00:18 2004] [error] child process 2129 still did not 
exit, sending a SIGKILL
[Sat Dec 18 13:00:18 2004] [error] child process 2130 still did not 
exit, sending a SIGKILL

and on and on and on...

So, apparently, Apache is having a problem and taking down the server.  
I eventually also see complaints about user 80 exceeding the 
kern.maxfiles limit.  That's probably when the server really takes a 
dump.

I've been monitoring top periodically to see if I can spot the problem, 
and an httpd process was consuming 95% of the cpu just now, and sure 
enough the above messages were streaming through the log.  I also 
notice the following:

httpd in free(): warning: chunk is already free
httpd in free(): warning: chunk is already free
httpd in free(): warning: chunk is already free
httpd in free(): warning: chunk is already free
httpd in free(): warning: chunk is already free
httpd in free(): warning: chunk is already free
httpd in free(): warning: chunk is already free


Now, my problem is I'm not sure how to find the source of this problem 
and stop it.  A google search on those log entries suggests that it may 
be an attempt to exploit the Chunk Handling Vulnerability, but my 
Apache is newer than the fix for that.

http://httpd.apache.org/info/security_bulletin_20020617.txt

Anyhow, can anyone give me a suggestion on how to troubleshoot this?  
Thanks!

Here is the Apache in question:

Server Version: Apache/1.3.33 (Unix) mod_ssl/2.8.22 OpenSSL/0.9.7e 
PHP/4.3.10 DAV/1.0.3



More information about the freebsd-questions mailing list