Trick for Apache2 and MySQL Crashing on High Load / Apache Benchmark
August 23, 2014
In the past, I have given some thought to a high load but not very much. For the most part having Lighttpd and Varnish in front of my Web servers alleviates most problems. That is until the other day when I decided to do an Apache benchmark test on my system and was able to bring down the server. The load made MySQL go insane by inflating it to 200% CPU usage! To make my servers available again I had to basically restart MySQL. This is not a huge issue, mainly because we do not have a high load but it was concerning.
I did a bunch of research online on how to tune MySQL and Apache, and most of them gave good advice which in turned lowered the memory footprint. But the benchmark would still take down the site. Finally I stumbled onto a parameter (thanks to an IRC buddy…and yes I still IRC). The solution was rather simple, but nobody really ever touches base on the subject. Mysql’s Query Cache This will basically cache all the Select queries so that when a page loads, if no data has changed, it will return the previously stored data. After changing that setting (set the query_cache_size = 64M ) I can benchmark the site and the MySQL does not crash. However, Lighttpd was slow to respond, it took some time for it to return the page (at least it returned the page!).
In the past I have heard of a mod for Lighttpd called mod_evasive. This mod was designed for DoS like attacks. What the mod will do is you are allowed to set a certain amount of IP connections that are allowed simultaneously to connect. I set my limit to 10, but some other sites may want a higher limit. So when an IP attempts to hit the site (page reload) or other too many times in a row, it sends out a 403 error to the IP and locks the IP out for a few seconds. After I enabled the mod_evasive, the Apache benchmark no longer took the system down.
So if you are having issues with your site and a high load, I would highly recommend the query_cache_size = 64M in the [mysqld] section of my.cnf. If you are experiencing issues from benchmarking your site, the mod_evasive will also assist in keeping your site above water from a single person attacking you. How will this hold up to a really high load? I honestly do not know as I have never been in that situation. I hope to be, but until I run into it, this will suffice for now.
Category: Lighttpd, Linux, MySQL, Varnishapache, assist, benchmark, cache, config, configuration, cpu, crash, crashing, evasion, evasive, fix, hault, high cpu, high load, lighttpd, Linux, mod, modify, mysqld, query, server, trick, usage, varnish, web, web server, webserver