Thread: Apache2 Optimizations for PHP based proxy

It took me a while to finally properly (IMO) optimize my Apache2 installs for proxies. With many websites you can get away with using the defaults configuration. But with proxies you tend to see a lot more server load than the average site, so you need to do much more optimization.

When someone goes to the average website, you get a request to the server, and then it either parses php or just outputs html. Rather standard. But with proxies the server sends requests to other websites, processes the information and translates all the links/images/scripts/etc on the website and then outputs that to the user. A lot more is going on.

The major configuration settings for Apache2 optimization lay in the mpm configuration. You can find the settings for this in apache2.conf in Debian. Most distributions have their own configuration, so you'll have to check your distro's specifics yourself. Do not ever use mpm_worker with proxies, or any PHP website. It's nice and threaded, but can cause a lot of problems when not serving static html sites.

You'll want to use mpm_prefork which is probably the default. Here's what the apache config comments say about prefork settings:
# StartServers: number of server processes to start
# MinSpareServers: minimum number of server processes which are kept spare
# MaxSpareServers: maximum number of server processes which are kept spare
# MaxClients: maximum number of server processes allowed to start
# MaxRequestsPerChild: maximum number of requests a server process serves

I'll go into more detail with about these.

StartServers is essentially just how many servers are started when the server starts. This isn't the most useful setting, it just helps keep your server from being immediately overloaded when (re)started.

MinSpareServers is the number of minimum unused servers to leave sitting around doing nothing, but waiting for connections.

MaxSpareServers is the maximum number of servers to leave sitting around doing nothing.

MaxClients is the maximum number of clients to have at all. This is a number that can get you in trouble a lot with a highly (ab)used server. Setting this too low will cause your server to essentially hang, as it will wait until there are free children to serve processes. Setting it too high will use up all your available memory and start swapping - or even use up both. Which will cause anything from a extremely slow server, to a full on hault. A simple formula to calculate a base for this is: (Total Memory - Operating System Memory) / Size Per Apache process.

MaxRequestsPerChild is how many requests are served by each child before they are killed, and a new one is generated. Another important setting as it will help you if there's any memory leaks, or any other trouble code. If something goes wrong in a child, it generally is killed off soon enough so it won't be an issue.

Here is an example of what I use on an Intel Core2Quad 9650 (3.00GHz) and 8GB RAM.

StartServers 10
MinSpareServers 10
MaxSpareServers 40
ServerLimit 1024
MaxClients 1024
MaxRequestsPerChild 500

Other important configuration settings are:

Timeout
KeepAlive
MaxKeepAliveRequests
KeepAliveTimeout

I like to keep all these low. Here are the settings I use:
Timeout 5
KeepAlive On
KeepAliveTimeout 5
MaxKeepAliveRequests 100

Bear in mind I've been using Linux for around 8 years now so I will probably be guilty of assuming too much. Any questions or corrections would be appreciated. I hope this helps some people out.

Edit: Ouch, simple spelling mistake in the title. Could a mod please fix? Also switch Glype with PHP based proxy.

A+ Thanks Nux

Interesting - I'm using similar settings but it all seems to be about testing what works. I've tried playing around with these settings but still after 12 hours I see over 2400 requests "hanging" - do you see similar things?

I guess I'll keep testing.

I used to let people download files through proxies, as long as they didn't reach the max file size (25M). I had this set to 0 $CONFIG['transfer_timeout']

I found that something changed in the last few months somewhere, either curl, apache, php, or something else.

If you set $CONFIG['transfer_timeout'] = 60; you will see processes hanging for 60 seconds all the time and dying. I'm not sure why this is. It happens on occasion when visiting Google.com.

I suggest you load up mod_status, with ExtendedStatus On and then goto your status page (a lot of times it's your.domain.com/server-status) and watch what's going on. If you see them sitting at W (sending reply) for long periods of time (thousands of seconds) then I'd be willing to be the transfer_timeout is your problem.