nginx + Apache2 Reverse Proxy: scaling your php5 site UP

nginx overview

Richweb uses the nginx web server in a reverse proxy configuration with apache2 to help our large, popular php based customer websites scale. The nginx software is configured to serve all static pages itself, and forward (reverse proxy) the http calls for the dynamic content to a local apache2 + mod_php process. Static files are typically set to expire in 24 to 48 hrs as well, so the local browsers will cache the content once downloaded once from the site.

In this configuration a busy site that may need 200 Apache child processes to run direct on apache2 instead needs about 25% of the number of child processes typically. If each child process needs 50Megs of RAM, the difference between 200 and 50 child processes is huge, in terms of memory and possible extra hardware costs that our clients may otherwise have to purchase.

There are a couple of things to note when your site is running in a reverse proxy configuration.

1. The apache config file (/etc/apache2/apache.conf) will list private/local ip addresses, such as 127.0.0.1, 192.168.1.1 or 192.0.192.1 instead of the real ip address or hostname that the website actually runs on.

2. The public or route-able ip addresses / hostnames will be listed in the nginx config file:
/etc/nginx/nginx.conf. Each website will have its own server {} section in nginx and a separate local ip and a matching vhost section in the apache conf.

3. SSL processing is handled by the front end proxy - nginx. This allows an http and https version of a website to be handled by the same apache2 backend vhost. An nginx redirect can be installed to make sure that the protected piece of the site is redirected:

        rewrite  ^/secure/(.*)$  https://www.mysite.org/secure/$1  permanent;
        rewrite  ^/secure/       https://www.mysite.org/secure/    permanent;

4. Gzip (compression) is handled in the nginx reverse proxy. compression can be enabled or disabled on a virtual host (site) basis. These are the nginx and cache settings we use typically for best performance:

    gzip  on;
    gzip_http_version 1.0;
    gzip_comp_level 2;
    gzip_proxied any;
    gzip_min_length  1100;
    gzip_buffers 16 8k;
    gzip_types text/plain text/html text/css application/x-javascript \
               text/xml application/xml application/xml+rss text/javascript \
               image/gif image/jpeg image/png;

    gzip_disable "MSIE [1-6].(?!.*SV1)";
    # Set a vary header so downstream proxies don't send cached gzipped
    # content to IE6
    gzip_vary on;

5. The ip address of the remote client (user browser) that will be seen by the apache2 + php process MAY be the ip address of the nginx webserver itself. This can cause problems for some web software that does not understand this configuration. We make sure in nginx to set the header
X-Real-IP to the actual ip address of the remote client:

proxy_set_header Host            $host;
proxy_set_header X-Real-IP       $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

So you may want to adjust software that looks at $REMOTE_ADDR

We install the apache module rpaf:
libapache2-mod-rpaf

that should take care of this problem by setting the $REMOTE_ADDR correctly inside the php process.

How does nginx speed up the website exactly?

With nginx handling SSL, static files (images, static .html files, javascript resources, CSS, etc) and compression this frees the backend apache and php process to just worry about the application code processing. If a page has 40 images and 10 CSS and javascript resources included then instead of apache having to handle 51 connections (50 static files plus the page itself) then apache only has to handle 1 connection and nginx will handle the other 50 without bothering apache. This allows apache to focus on caching the application code (apc cache uses a lot of memory) as well so that the php does not have to be compiled on each page view.

With nginx in place apache can scale to handle 1 simultaneous user per apache child process. Without nginx on many sites the ratio would be more like 1 simultaneous user requiring 4 or 5 apache child processes.

So, the bottom line is that nginx can help scale your site by a factor of 4 or 5 in many cases, which is an outstanding improvement.