Wordpress And High disk i/o

Question

So i run a Wordpress BLOG with the following setup:

1.Two dedicated servers - one for application and one for database Application server specs: 32cores 126gb ram 4tb hdd running centos 7 Database Server specs : 16cores 140gb ram 3tb hdd running centos 7

I have optimized the code to a good page speed score but i have been experiencing connection time outs, when my blog traffic reaches 1000 active users.

currently my page size is 3.9mb and makes 250 requests per page load.

i have however employed a caching plugin, w3 total cache which helps with the optimization. As a result of the time outs i have noticed that i have a very high disk i/o when this occurs, yet i have employed apc and memcached systems on my server.

On further investigations of the top processes writing to my hard disk i found out that php-cgi occupies the bulk of the process list both on htop and top result screens.

my question: how can i manage the high disk i/o to a reasonable level, or how can i fine tune php-cgi to utilize my harddisk space more efficiently.

Answer 1

so i managed to figure out what the problem was, your inputs were all help full in pointing me in the right direction. so what was causing my high disk i/o was the php_cgi_maxrequest value which was set at 99999, so i increased the value to 200,000 in the disk i/o shot down drastically, and i'm experiencing a more stable server environment since the change.

Answer 2

I'm not sure if it's the cause, but wouldn't you be better off using mod_php? The performance should be much better than php-cgi - I've never used that in production, but I'm a creature of habit. PHP shouldn't be writing a lot to the disk. AFAIK, the main cases where it would write to the disk are.

1. Server is using swap space (unlikely due to the large amount of RAM you have - worth checking though)
2. PHP is generating hundreds of notices and warnings and writing to the error logs (solution: change error reporting level or actually fix the code problems)
3. W3 Total Cache config problem - try altering the expiration time of the cache to see if it makes a difference.

3.9MB is also a crazy page size, you could probably do a lot to optimise that, use a CDN for static assets etc.

Answer 3

writes probably have nothing to do with that (unless there are more details), your problem is file reads

An obvious cause is using cgi instead of mod_php or fastcgi https://stackoverflow.com/questions/14489346/is-it-possible-to-run-apc-with-php-cgi. The use of CGI makes the PHP interpreter to read all the files for each request, and if APC could have worked, the PHP would have read only about once every two seconds (configurable), and this is before considering other APC optimizations.

It doesn't help that you have so many requests per page. Fuck whatever google speed tells you about your site, it sends you to optimize browser caching but in practice the effectiveness of browser caching depends on how frequently the people come to your site. IIRC a test done by yahoo several years ago showed that 50% of visitors did not had the yahoo files cached when they accessed the site. The point is that high score in browser caching do not translate to your server not being hit with 250 request for every page request.

Depending on the rate of page requests and hard drive technology you might just be putting the disk controller at busy state all the time (this will also depend on how much the hard drive driver caches files in memory. If you have to have so many requests, maybe you should put the relevant files on memory disk.

You have a beast of an app server and 1000 user at the "same time" is not that much (I saw 2x8GB handling 7x your load without sweating at all). Of cource it depends on what your code actually does.... My suggestion is that you should investigate the cost of using CDN. It might be that once your hardware do not need to serve all those files you might be able to reduce the server requirements which might offset the cost of the CDN.