bennett haselton

I'm running a Web server that runs a lot of popular PHP and CGI
scripts and I'm trying to prevent the requests from swamping the
machine and making in unresponsive. I am aware, of course, that with
a given script and with fixed hardware resources, there's an upper
limit on how many requests you can serve. I just want to make httpd
turn down requests gracefully when it hits that limit, instead of the
whole machine slowing to a crawl and nobody's requests being served at
all.

It's acceptable if a certain number of http requests get cut off; for
example I even tried setting up cron jobs to just restart the Web
server every 10 minutes:
[root@sls-ce3p12 ~]# crontab -l
00 * * * * service httpd restart
10 * * * * service httpd restart
20 * * * * service httpd restart
30 * * * * service httpd restart
40 * * * * service httpd restart
50 * * * * service httpd restart

However that didn't work -- "ps auwx" shows lots of httpd processes
that have been running for more than 10 minutes:

[partial output -- truncated by the top of the ssh window]
root 2414 0.0 1.3 16780 6768 ? Ss 12:36 0:00 /usr/
sbin/httpd
root 2423 0.0 0.1 6516 572 ? Ss 12:36 0:00 crond
root 2432 0.0 0.0 2328 296 ? SNs 12:36 0:00 anacron
-s
root 2440 0.0 0.0 3088 324 ? Ss 12:36 0:00 /usr/
sbin/atd
dbus 2449 0.0 0.0 13832 376 ? Ssl 12:36 0:00 dbus-
daemon-1 --s
ystem
apache 2461 1.0 2.2 22056 11340 ? S 12:36 0:11 /usr/
sbin/httpd
root 2468 0.0 0.1 6508 908 ? Ss 12:36 0:00 hald
root 2477 0.0 0.0 3088 348 tty1 Ss+ 12:36 0:00 /sbin/
mingetty tt
y1
root 2478 0.0 0.0 3052 348 tty2 Ss+ 12:36 0:00 /sbin/
mingetty tt
y2
root 2479 0.0 0.0 1596 348 tty3 Ss+ 12:36 0:00 /sbin/
mingetty tt
y3
root 2480 0.0 0.0 1940 348 tty4 Ss+ 12:36 0:00 /sbin/
mingetty tt
y4
root 2481 0.0 0.0 2252 348 tty5 Ss+ 12:36 0:00 /sbin/
mingetty tt
y5
root 2482 0.0 0.0 1644 348 tty6 Ss+ 12:36 0:00 /sbin/
mingetty tt
y6
apache 2485 0.9 2.0 21604 10392 ? D 12:36 0:09 /usr/
sbin/httpd
apache 2706 1.1 1.9 23240 9756 ? S 12:36 0:11 /usr/
sbin/httpd
apache 2810 1.0 2.1 24500 11260 ? S 12:36 0:11 /usr/
sbin/httpd
apache 2824 1.4 2.0 21408 10292 ? S 12:36 0:15 /usr/
sbin/httpd
apache 3067 1.2 2.2 23520 11772 ? S 12:36 0:13 /usr/
sbin/httpd
apache 3069 1.3 2.1 24508 10796 ? S 12:36 0:13 /usr/
sbin/httpd
apache 3070 1.5 2.8 24380 14464 ? D 12:36 0:16 /usr/
sbin/httpd
apache 3073 1.1 1.6 21524 8680 ? S 12:36 0:11 /usr/
sbin/httpd
apache 3074 1.3 2.0 24584 10564 ? S 12:36 0:13 /usr/
sbin/httpd
apache 3083 1.0 1.7 21480 8756 ? S 12:37 0:10 /usr/
sbin/httpd
apache 3097 1.0 1.6 26780 8668 ? R 12:37 0:10 /usr/
sbin/httpd
apache 3124 1.6 2.0 23628 10568 ? S 12:37 0:16 /usr/
sbin/httpd
apache 3125 1.3 1.7 21376 8768 ? D 12:37 0:13 /usr/
sbin/httpd
apache 3135 0.8 2.0 21484 10244 ? D 12:37 0:08 /usr/
sbin/httpd
apache 3146 0.7 1.9 23044 10124 ? R 12:38 0:07 /usr/
sbin/httpd
apache 3153 1.1 2.1 22936 11108 ? S 12:38 0:10 /usr/
sbin/httpd
apache 3159 1.3 1.7 23836 8712 ? S 12:38 0:12 /usr/
sbin/httpd
apache 3160 1.5 2.6 28812 13428 ? R 12:38 0:14 /usr/
sbin/httpd
apache 3165 1.0 1.6 23336 8332 ? S 12:38 0:09 /usr/
sbin/httpd
apache 3166 1.6 1.8 25496 9284 ? R 12:38 0:15 /usr/
sbin/httpd
apache 3193 0.5 1.6 21956 8592 ? S 12:39 0:04 /usr/
sbin/httpd
apache 3198 0.7 1.7 22044 8780 ? S 12:39 0:07 /usr/
sbin/httpd
apache 3200 1.2 1.6 22732 8652 ? D 12:39 0:10 /usr/
sbin/httpd
root 3203 0.0 0.2 8408 1356 ? Ss 12:39 0:00 sshd:
root@pts/0
apache 3206 1.2 1.9 21344 9852 ? S 12:39 0:11 /usr/
sbin/httpd
apache 3218 1.0 1.7 23400 8708 ? S 12:39 0:09 /usr/
sbin/httpd
apache 3219 1.1 1.6 21344 8568 ? S 12:39 0:09 /usr/
sbin/httpd
apache 3220 0.8 2.0 21684 10500 ? S 12:39 0:07 /usr/
sbin/httpd
apache 3222 1.0 1.6 21344 8368 ? S 12:39 0:09 /usr/
sbin/httpd
apache 3225 1.5 2.1 23544 10872 ? S 12:39 0:13 /usr/
sbin/httpd
apache 3227 1.2 1.6 21360 8644 ? S 12:39 0:11 /usr/
sbin/httpd
apache 3228 0.9 1.8 22060 9612 ? S 12:39 0:07 /usr/
sbin/httpd
root 3240 0.0 0.1 3952 740 ? Ss 12:39 0:00 /usr/
libexec/open
ssh/sftp-server
root 3241 0.0 0.2 5632 1072 pts/0 Ss 12:39 0:00 -bash
apache 3272 1.2 1.7 23056 8704 ? R 12:40 0:10 /usr/
sbin/httpd
apache 3342 0.9 2.7 22748 14136 ? R 12:41 0:07 /usr/
sbin/httpd
apache 3351 1.0 1.9 21336 10052 ? S 12:41 0:07 /usr/
sbin/httpd
apache 3364 0.5 2.7 23316 13860 ? S 12:42 0:03 /usr/
sbin/httpd
apache 3366 0.8 2.0 21968 10456 ? S 12:42 0:06 /usr/
sbin/httpd
apache 3368 0.5 1.7 22636 8900 ? S 12:42 0:03 /usr/
sbin/httpd
apache 3369 0.7 1.6 21320 8204 ? S 12:42 0:05 /usr/
sbin/httpd
apache 3371 1.5 1.9 21516 10024 ? S 12:42 0:11 /usr/
sbin/httpd
apache 3373 0.7 2.3 21184 12280 ? S 12:42 0:05 /usr/
sbin/httpd
apache 3401 0.5 1.9 21408 9908 ? S 12:43 0:03 /usr/
sbin/httpd
apache 3403 0.3 1.5 24212 8108 ? S 12:43 0:02 /usr/
sbin/httpd
apache 3406 0.8 1.7 23180 8836 ? R 12:43 0:05 /usr/
sbin/httpd
apache 3423 0.1 1.5 21404 7900 ? R 12:43 0:00 /usr/
sbin/httpd
apache 3428 0.1 2.1 20092 10944 ? R 12:43 0:00 /usr/
sbin/httpd
apache 3460 1.2 1.6 22016 8692 ? S 12:44 0:07 /usr/
sbin/httpd
apache 3479 0.9 2.0 22188 10488 ? S 12:44 0:05 /usr/
sbin/httpd
apache 3482 1.4 1.9 21232 9872 ? S 12:44 0:08 /usr/
sbin/httpd
apache 3484 0.4 1.3 18836 7032 ? S 12:44 0:02 /usr/
sbin/httpd
apache 3492 0.9 1.4 17904 7644 ? S 12:44 0:05 /usr/
sbin/httpd
apache 3493 0.1 1.7 21244 9140 ? S 12:45 0:00 /usr/
sbin/httpd
apache 3496 2.0 2.0 21456 10700 ? S 12:45 0:11 /usr/
sbin/httpd
apache 3498 1.2 2.5 21304 12824 ? S 12:45 0:06 /usr/
sbin/httpd
apache 3499 0.6 2.6 23332 13800 ? S 12:45 0:03 /usr/
sbin/httpd
apache 3500 0.8 2.5 21316 12876 ? S 12:45 0:04 /usr/
sbin/httpd
apache 3501 0.1 1.6 21404 8276 ? R 12:45 0:00 /usr/
sbin/httpd
apache 3502 0.1 1.7 21408 8704 ? R 12:45 0:00 /usr/
sbin/httpd
apache 3505 1.3 2.0 21828 10452 ? S 12:45 0:07 /usr/
sbin/httpd
apache 3506 0.6 1.9 21292 9988 ? S 12:45 0:03 /usr/
sbin/httpd
apache 3508 0.8 1.9 21660 10160 ? S 12:45 0:04 /usr/
sbin/httpd
apache 3509 1.7 2.8 22916 14556 ? R 12:45 0:09 /usr/
sbin/httpd
apache 3510 1.1 2.4 21240 12376 ? S 12:45 0:06 /usr/
sbin/httpd
apache 3512 0.5 1.6 21732 8536 ? S 12:45 0:03 /usr/
sbin/httpd
apache 3513 1.0 1.9 22632 10180 ? S 12:45 0:05 /usr/
sbin/httpd
apache 3515 0.1 1.2 18324 6288 ? R 12:45 0:00 /usr/
sbin/httpd
apache 3518 0.1 2.0 22068 10436 ? R 12:45 0:00 /usr/
sbin/httpd
apache 3524 1.0 1.7 22200 9016 ? S 12:45 0:05 /usr/
sbin/httpd
apache 3525 0.1 1.2 18344 6620 ? R 12:45 0:00 /usr/
sbin/httpd
apache 3526 0.1 1.7 21404 9068 ? R 12:45 0:00 /usr/
sbin/httpd
apache 3528 0.6 1.3 17904 6988 ? S 12:45 0:03 /usr/
sbin/httpd
apache 3529 0.8 1.6 21444 8500 ? S 12:45 0:04 /usr/
sbin/httpd
apache 3545 1.2 1.6 21264 8452 ? S 12:46 0:06 /usr/
sbin/httpd
apache 3551 1.4 1.8 21508 9228 ? S 12:46 0:06 /usr/
sbin/httpd
apache 3554 0.2 1.3 18692 6820 ? R 12:46 0:01 /usr/
sbin/httpd
apache 3556 0.9 2.3 22984 12024 ? S 12:46 0:04 /usr/
sbin/httpd
apache 3560 1.0 1.6 21324 8216 ? S 12:46 0:04 /usr/
sbin/httpd
apache 3562 0.1 1.2 18612 6588 ? R 12:46 0:00 /usr/
sbin/httpd
apache 3563 0.3 1.3 17548 6936 ? S 12:46 0:01 /usr/
sbin/httpd
apache 3566 1.2 1.6 23548 8476 ? S 12:46 0:05 /usr/
sbin/httpd
apache 3569 0.3 2.7 23368 14112 ? S 12:46 0:01 /usr/
sbin/httpd
apache 3570 0.1 1.5 21368 7932 ? R 12:46 0:00 /usr/
sbin/httpd
apache 3572 0.1 1.9 21216 10000 ? S 12:46 0:00 /usr/
sbin/httpd
apache 3573 0.2 1.2 18692 6396 ? R 12:46 0:01 /usr/
sbin/httpd
apache 3574 0.1 1.9 21172 9960 ? S 12:46 0:00 /usr/
sbin/httpd
apache 3575 0.1 2.4 22772 12396 ? R 12:46 0:00 /usr/
sbin/httpd
apache 3576 0.4 1.3 17612 7004 ? S 12:46 0:01 /usr/
sbin/httpd
apache 3577 0.4 0.0 0 0 ? Z 12:46 0:02 [httpd]
<defunct>
apache 3578 0.1 1.2 18456 6288 ? R 12:46 0:00 /usr/
sbin/httpd
apache 3579 0.1 1.5 21368 8092 ? R 12:46 0:00 /usr/
sbin/httpd
apache 3580 0.1 2.0 21348 10292 ? S 12:46 0:00 /usr/
sbin/httpd
apache 3581 0.1 1.8 21408 9496 ? R 12:46 0:00 /usr/
sbin/httpd
apache 3603 0.1 1.2 18640 6480 ? R 12:48 0:00 /usr/
sbin/httpd
apache 3604 0.1 1.8 21532 9656 ? R 12:48 0:00 /usr/
sbin/httpd
apache 3606 1.0 2.0 22004 10304 ? S 12:48 0:03 /usr/
sbin/httpd
apache 3607 0.3 2.4 21220 12612 ? S 12:48 0:01 /usr/
sbin/httpd
apache 3608 1.0 2.5 21488 13060 ? S 12:48 0:03 /usr/
sbin/httpd
apache 3609 0.6 1.4 17972 7372 ? S 12:48 0:02 /usr/
sbin/httpd
apache 3610 0.1 1.2 18228 6536 ? R 12:48 0:00 /usr/
sbin/httpd
apache 3611 1.4 1.6 21480 8632 ? S 12:48 0:05 /usr/
sbin/httpd
apache 3612 0.2 2.0 21932 10592 ? R 12:48 0:00 /usr/
sbin/httpd
apache 3613 0.7 2.4 21288 12412 ? S 12:48 0:02 /usr/
sbin/httpd
apache 3614 0.1 2.3 21240 12140 ? S 12:48 0:00 /usr/
sbin/httpd
apache 3616 1.0 2.4 21412 12720 ? S 12:48 0:03 /usr/
sbin/httpd
apache 3617 0.2 2.5 22732 13268 ? R 12:48 0:00 /usr/
sbin/httpd
apache 3618 0.1 1.3 17724 7036 ? S 12:48 0:00 /usr/
sbin/httpd
apache 3619 0.9 2.4 21432 12756 ? S 12:48 0:03 /usr/
sbin/httpd
apache 3620 1.9 2.9 23592 15244 ? S 12:48 0:06 /usr/
sbin/httpd
apache 3622 1.7 1.8 21524 9300 ? S 12:48 0:06 /usr/
sbin/httpd
apache 3624 0.7 1.3 18880 7128 ? R 12:48 0:02 /usr/
sbin/httpd
apache 3625 0.1 1.8 21404 9680 ? R 12:48 0:00 /usr/
sbin/httpd
apache 3626 0.0 1.2 17376 6440 ? S 12:48 0:00 /usr/
sbin/httpd
apache 3668 0.1 1.2 18112 6364 ? R 12:49 0:00 /usr/
sbin/httpd
apache 3669 0.1 1.2 18164 6424 ? R 12:49 0:00 /usr/
sbin/httpd
apache 3670 1.1 2.4 21180 12556 ? S 12:49 0:02 /usr/
sbin/httpd
apache 3672 0.0 1.2 17208 6420 ? S 12:49 0:00 /usr/
sbin/httpd
apache 3673 0.0 1.2 17208 6424 ? S 12:49 0:00 /usr/
sbin/httpd
root 3677 0.0 0.1 7096 824 ? S 12:50 0:00 crond
root 3698 0.0 0.0 3492 436 ? Rs 12:51 0:00 /bin/sh
-c servic
e httpd restart
apache 3699 0.2 2.1 20232 11100 ? R 12:51 0:00 /usr/
sbin/httpd
apache 3701 0.0 1.2 17288 6580 ? S 12:51 0:00 /usr/
sbin/httpd
apache 3705 0.0 0.4 5208 2352 ? R 12:51 0:00 /usr/
bin/perl /va
r/www/cgi-bin/index.cgi
apache 3706 0.0 0.4 4496 2332 ? R 12:52 0:00 /usr/
bin/perl /va
r/www/cgi-bin/index.cgi
apache 3708 0.1 0.4 4680 2304 ? R 12:52 0:00 /usr/
bin/perl /va
r/www/cgi-bin/index.cgi
apache 3710 0.1 0.4 5972 2096 ? R 12:52 0:00 /usr/
bin/perl /va
r/www/cgi-bin/index.cgi
apache 3713 0.1 0.4 4852 2284 ? R 12:52 0:00 /usr/
bin/perl /va
r/www/cgi-bin/index.cgi
apache 3716 3.2 2.3 21328 12160 ? S 12:53 0:01 /usr/
sbin/httpd
apache 3720 3.7 2.6 21908 13384 ? S 12:53 0:02 /usr/
sbin/httpd
apache 3725 0.3 1.2 17220 6460 ? S 12:53 0:00 /usr/
sbin/httpd
apache 3726 0.3 1.2 17208 6392 ? S 12:53 0:00 /usr/
sbin/httpd
apache 3727 0.2 1.2 17208 6448 ? S 12:53 0:00 /usr/
sbin/httpd
apache 3731 0.2 1.2 17128 6428 ? S 12:53 0:00 /usr/
sbin/httpd
apache 3732 0.2 1.2 17144 6468 ? S 12:53 0:00 /usr/
sbin/httpd
apache 3733 0.2 1.2 17204 6428 ? S 12:53 0:00 /usr/
sbin/httpd
apache 3738 0.2 0.3 4636 1760 ? R 12:54 0:00 /usr/
bin/perl /va
r/www/cgi-bin/index.cgi
root 3743 0.0 0.1 3244 788 pts/0 R+ 12:54 0:00 ps auwx
apache 3745 0.0 0.3 5492 1736 ? R 12:54 0:00 /usr/
bin/perl /va
r/www/cgi-bin/index.cgi
apache 3747 0.0 0.3 4032 1712 ? R 12:54 0:00 /usr/
bin/perl /va
r/www/cgi-bin/index.cgi
apache 3748 0.0 0.3 4112 1752 ? R 12:54 0:00 /usr/
bin/perl /va
r/www/cgi-bin/index.cgi
apache 3751 0.0 0.3 4196 1676 ? R 12:54 0:00 /usr/
bin/perl /va
r/www/cgi-bin/index.cgi
apache 3753 0.0 0.3 4256 1688 ? R 12:54 0:00 /usr/
bin/perl /va
r/www/cgi-bin/index.cgi
apache 3757 0.0 0.2 5308 1520 ? R 12:54 0:00 /usr/
bin/perl /va
r/www/cgi-bin/capture.cgi
apache 3759 0.0 0.3 3924 1640 ? R 12:55 0:00 /usr/
bin/perl /va
r/www/cgi-bin/index.cgi
apache 3763 0.0 0.3 5152 1616 ? R 12:55 0:00 /usr/
bin/perl /va
r/www/cgi-bin/capture.cgi
apache 3764 0.0 0.2 3676 1500 ? R 12:55 0:00 /usr/
bin/perl /va
r/www/cgi-bin/capture.cgi

Other than "buy more memory", "optimize your script" (it's a third-
party script), etc., how can I prevent this?

There is a section in httpd.conf that reads:

# prefork MPM
# StartServers: number of server processes to start
# MinSpareServers: minimum number of server processes which are kept
spare
# MaxSpareServers: maximum number of server processes which are kept
spare
# ServerLimit: maximum value for MaxClients for the lifetime of the
server
# MaxClients: maximum number of server processes allowed to start
# MaxRequestsPerChild: maximum number of requests a server process
serves
<IfModule prefork.c>
StartServers 8
MinSpareServers 5
MaxSpareServers 20
ServerLimit 256
MaxClients 256
MaxRequestsPerChild 4000
</IfModule>


Is that the best way to fix things, entering a lower value for
ServerLimit and MaxClients for example? How can I tell if the
<IfModule prefork.c> section is even being parsed -- how can I tell if
that module is in effect or loaded or whatever? And then what will
users see if they try to make more requests than the machine can
handle, do they get a "server too busy" error page?

nick kew

On Wed, 12 Sep 2007 11:13:54 -0700


http://svn.in.force-elite.com:8080/s...load_average.c

Use it to return "server busy" for heavy-duty CGI and PHP scripts
when load average exceeds some threshold. You can still continue
to serve static pages.

--
Nick Kew

Application Development with Apache - the Apache Modules Book
http://www.apachetutor.org/

bennett haselton

Thanks. But isn't there also a built-in way to do this in Apache? To
give a "HTTP/1.1 Server Too Busy" page if load average is too high?

If possible, I want to find a solution that involves installing as
little extra stuff as possible, since I will have the make the changes
on many machines. I tried googling "HTTP/1.1 Server Too Busy" but all
the matches are about people complaining they're getting that error
from somewhere

Is the "HTTP/1.1 Server Too Busy" in fact what you get if you set
MaxClients to some low number and the number of client requests
exceeds that number?