security - Apache configuration: MaxClients reached | Many unknown GETs in access_log

Question

I am setting up a new apache+mysql server. It has only 3 websites and it really does not have much activity. I mostly use it for programming and testing.

The httpd.conf of the server is this one:

....

<IfModule prefork.c>
StartServers       8
MinSpareServers    5
MaxSpareServers    20
ServerLimit       256
MaxClients        256
MaxRequestsPerChild  4000
</IfModule>

<IfModule worker.c>
StartServers         4
MaxClients         300
MinSpareThreads     25
MaxSpareThreads     75
ThreadsPerChild     25
MaxRequestsPerChild 0
</IfModule>

....

For some reason as soon as I start the server and visit a page (even if it is really basic, with no connections to the database or whatsoever.. I get this:

[Wed Dec 11 13:59:10 2013] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Wed Dec 11 13:59:10 2013] [notice] Digest: generating secret for digest authentication ...
[Wed Dec 11 13:59:10 2013] [notice] Digest: done
[Wed Dec 11 13:59:10 2013] [notice] Apache/2.2.15 (Unix) DAV/2 PHP/5.3.3 configured -- resuming normal operations
[Wed Dec 11 13:59:25 2013] [error] server reached MaxClients setting, consider raising the MaxClients setting

If I execute "ps -ef"as soon as I start the server, I see all this processes running:

UID        PID  PPID  C STIME TTY          TIME CMD
....
root      2945     1  2 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2947  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2948  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2949  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2950  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2951  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2952  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2953  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2954  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2955  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2956  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2957  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2958  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2959  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2960  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2961  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2962  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2963  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2964  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2965  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2966  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2967  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2968  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2969  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2970  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2971  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2972  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2973  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2974  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2975  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2976  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2977  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2978  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2979  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2980  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2981  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2982  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2983  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2984  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2985  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2986  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2987  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2988  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2989  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2990  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2991  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2992  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2993  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2994  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2995  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2996  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2997  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2998  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    2999  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3000  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3001  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3002  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3003  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3004  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3005  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3006  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3007  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3008  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3009  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3010  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3011  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3012  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3013  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3014  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3015  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3016  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3017  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3018  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3019  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3020  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3021  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3022  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3023  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3024  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3025  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3026  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3027  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3028  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3029  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3030  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3031  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3032  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3033  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3034  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3035  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3036  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3037  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3038  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3039  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3040  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3041  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3042  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3043  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3044  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3045  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3046  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3047  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3048  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3049  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3050  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3051  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3052  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3053  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3054  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3055  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3056  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3057  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3058  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3059  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3060  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3061  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3062  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3063  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3064  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3065  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3066  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3067  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3068  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3069  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3070  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3071  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3072  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3073  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3074  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3075  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3076  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3077  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3078  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3079  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3080  2945  0 14:08 ?        00:00:00 /usr/sbin/httpd
apache    3081  2945  0 14:09 ?        00:00:00 /usr/sbin/httpd
apache    3082  2945  0 14:09 ?        00:00:00 /usr/sbin/httpd
apache    3083  2945  0 14:09 ?        00:00:00 /usr/sbin/httpd
apache    3084  2945  0 14:09 ?        00:00:00 /usr/sbin/httpd
apache    3085  2945  0 14:09 ?        00:00:00 /usr/sbin/httpd
apache    3086  2945  0 14:09 ?        00:00:00 /usr/sbin/httpd
apache    3087  2945  0 14:09 ?        00:00:00 /usr/sbin/httpd
apache    3088  2945  0 14:09 ?        00:00:00 /usr/sbin/httpd
apache    3089  2945  0 14:09 ?        00:00:00 /usr/sbin/httpd
apache    3090  2945  0 14:09 ?        00:00:00 /usr/sbin/httpd
apache    3091  2945  0 14:09 ?        00:00:00 /usr/sbin/httpd
apache    3092  2945  0 14:09 ?        00:00:00 /usr/sbin/httpd
apache    3093  2945  0 14:09 ?        00:00:00 /usr/sbin/httpd
apache    3094  2945  0 14:09 ?        00:00:00 /usr/sbin/httpd
apache    3095  2945  0 14:09 ?        00:00:00 /usr/sbin/httpd
apache    3096  2945  0 14:09 ?        00:00:00 /usr/sbin/httpd
apache    3097  2945  0 14:09 ?        00:00:00 /usr/sbin/httpd
apache    3098  2945  0 14:09 ?        00:00:00 /usr/sbin/httpd
apache    3099  2945  0 14:09 ?        00:00:00 /usr/sbin/httpd
apache    3100  2945  0 14:09 ?        00:00:00 /usr/sbin/httpd
apache    3101  2945  0 14:09 ?        00:00:00 /usr/sbin/httpd
apache    3102  2945  0 14:09 ?        00:00:00 /usr/sbin/httpd
apache    3103  2945  0 14:09 ?        00:00:00 /usr/sbin/httpd
apache    3104  2945  0 14:09 ?        00:00:00 /usr/sbin/httpd
apache    3105  2945  0 14:09 ?        00:00:00 /usr/sbin/httpd
apache    3106  2945  0 14:09 ?        00:00:00 /usr/sbin/httpd
apache    3107  2945  0 14:09 ?        00:00:00 /usr/sbi

Answer 1

I could finally fix this yesterday. The problem was that my server was acting as a open proxy.

The entries displayed in the access_log are usually the result of malicious clients trying to exploit open proxy servers to access a website without revealing their true location. They could be doing this to manipulate pay-per-click ad systems, to add comment or link-spam to someone else's site, or just to do something nasty without being detected.

How did I prevent these requests from accessing the foreign server through my server?

First, if you don't need to run a proxy server, disable mod_proxy by commenting out its LoadModule line or setting ProxyRequests off in httpd.conf. Remember that disabling ProxyRequests does not prevent you from using a reverse proxy with the ProxyPass directive.

I didn't like the idea of my server responding to requests for random hostnames.

You can configure Apache to deny access to any host that isn't specifically configured by setting up a default virtual host:

NameVirtualHost *:80

<VirtualHost *:80>
  ServerName default.only
  <Location />
    Order allow,deny
    Deny from all
  </Location>
</VirtualHost>

<VirtualHost *:80>
  ServerName realhost1.example.com
  ServerAlias alias1.example.com alias2.example.com
  DocumentRoot /path/to/site1
</VirtualHost>

After these changes, you can try yourself to use your server as a proxy to access other sites and make sure that you get either a failure, or local content from your site. Among the ways to do this:

Configure your browser to use your web server as its default proxy server and then try to request foreign sites. You should get only your own website content back in reply. Manually construct requests using telnet:

telnet yoursite.example.com 80
GET http://www.yahoo.com/ HTTP/1.1
Host: www.yahoo.com