Take the 2-minute tour ×
WordPress Development Stack Exchange is a question and answer site for WordPress developers and administrators. It's 100% free, no registration required.

I had a close look at Wordpress memory consumption. On my site, it seems that for each page hit 20MB of RAM gets allocated, just to prepare comfy environment for all the plugins to run in. I plotted it thusly:

There is no single spot to optimize, no single bad guy who eats most of the memory. Consumption is all spread over many many many php modules.

How can we make Wordpress initialize its environment in memory only once, and then reuse it many times for each hit? I do not want to have slow PHP eat 20 MB at each user click - even on a server with lots of memory, it takes seconds to get all that work done. You would basically need read-only chunks of memory that can be reused.

Also... why 20MB? Can anyone provide insights into this?

Edit: Here is WinCacheGrind output on the Wordpress running on my development machine (vastly faster than the shared hosting). As you can see, it takes over a second of crunching just to produce the main page's HTML. Slow that down by shared hosting and you have a recipe for trouble. I picked the method that takes most of the time. How would you go about optimizing this?

Edit: Here are query statistics from this fantastic functions.php profiling tool.

Load: 12 queries - 532ms - 19.1MB - 43 cache hits / 53
Query: 15 queries - 563ms - 19.0MB - 72 cache hits / 86
Display: 21 queries - 705ms - 19.2MB - 234 cache hits / 257

Edit: Do you want to see something guaranteed to freak you out? Insert these lines at the end of index.php:


echo "<pre>\n";
print_r(get_defined_vars());
echo "</pre>\n";

I tried to count how many times is the current post's body stored in the memory. I counted 20 instances. Then I realized that PHP has reference counting, so the amount of copies reduced to just three: two seem to be in WP_Query, one in the object cache. I am investigating further.

This is why I think WordPress is in need of refactoring targeting the memory issues. You can no longer blame its memory consumption on sheer complexity of what it does. It simply does a bunch of things wrong.

Edit: After a day of trying to figure this out, here are my findings:

1) 88% of all memory comes from require or include or include_once type of calls:

2) The php file includes happen mostly during the first part of serving a request (not surprisingly), which is also where all the memory is eaten up:

3) It is quite interesting to plot all the functions that are being executed during making a request. There are over 12000 calls total. I jittered them to make it more visible (the Level axis is basically depth of stack):

4) The only way forward that I can think of is minimizing the amount of .php files included. If I split the functions per file they came from, you can see that many files are hit once or twice at most. We need a way how to skip those when they are not needed. For instance my remote database backup plugin gets loaded and registered, just to be never used at all. Here is the above plot split by file name:

I am offering a bounty worth all my reputation :) for refactorings that would lead to cutting my blogs memory footprint by 30% or more.

Edit: I installed WP 3.1, here is comparison with the old version.

Blue is WP 3.1, red is 3.0.4. The new WP is faster, but also eats more memory.

Here is a list by include file.

This let me realize just how much memory is eaten by "All In One SEO pack" - one avenue would be to use only a fraction of the plugin's functionality to get what I want. Also, my own plugins seem to be pretty bad.

I would like to try conditional loading on e.g. comment.php (I disallow comments on my blog) and several others. I deleted all the deprecated code. I trimmed down kses.php to only load its global tables on demand. I simplified l10n (I do no localization), making its functions return the strings right away, without lookups. I am still far from the 30% mark I arbitrarily set up.

Edit: I downloaded and enabled APC with default settings (32MB of opcode cache). Here is the comparison:

You can see that the code loading accelerated massively, and the code also takes less space in memory (probably because we deal just with opcodes, not the original source). The memory consumption is however still quite high.

share|improve this question

closed as not constructive by Rarst Oct 18 '12 at 23:12

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.If this question can be reworded to fit the rules in the help center, please edit the question.

    
Could you upload cachegrind file itself somewhere? Just note I don't remember if anything worth keeping private is included in it, if is - then don't. –  Rarst Feb 26 '11 at 17:39
    
@Rarst It should be fine. foxloft.com/files/mbala/cachegrind.out –  Roman Zenka Feb 26 '11 at 17:58
1  
hm, I agree with your conclusion - nothing really jumps out, asking to be fixed. I've dumped fresh profile on my local test stack (3.1, MS, Twenty Ten, Theme Unit Test data) and got 1,5s (most of difference seems to be because of custom menu - thing is slooow). So I guess nothing to fix = caching research it is. –  Rarst Feb 26 '11 at 18:55
    
@Rarst Thank you very much for your help. I do think that there is stuff to fix, but it would require changing the architecture of Wordpress to some completely different philosophy, and that is too much work. –  Roman Zenka Feb 26 '11 at 18:57
add comment

8 Answers 8

you probably won't manage to reduce ram usage that much. But if you're using mod_php, you may want to switch to mod_fcgid instead.

while mod_php is slightly slower, it loads php even when it doesn't need to, such as serving images, static files, or even caching. If you have lot of requests, this is lot of ram.

using fcgid will reduce this a lot.

also, using a static cache (like w3total cache) will avoid calling php at all which is a really great advantage: less ram usage, less db connections.

share|improve this answer
add comment

a few basic suggestions :

  1. w3 total cache plugin for caching..
  2. get memcache installed and enabled, also enable from w3 total cache settings (opcode cache is a good option too but it doesn't go well with w3 total cache plugin)
  3. minimize queries to direct links in theme files..
  4. Disable all extra unused plugins and remove.
  5. optimize Database.

i am running a well known wordpress site with huge traffic daily.. im not on dedicated even, doing great for me :)

share|improve this answer
add comment

Ha. I'm working on a web app now that I fully intend to overload with data and usage beyond what my shared hosting account can handle, so I decided - while it would have been super easy to build in WP - to try working from BackPress as a framework and build out only what I needed for my specific use-cases.

So I've been able to reduce my core environment from the hundreds of PHP files in WP to only the twenty or so that I actually need, while still being able to leverage all the db, HTTP, user-management, formatting, and cron functions that I love in WordPress.

The problem is that its a lot of work, and I would never trust my hackjob for anything beyond my own personal use. If you want to use the full WP environment, take it as it is. Its as good as it is because of hundreds of developers fine-tuning it over several years. Like everyone here has said, you'll get a lot farther by finding a better hosting plan and researching caching techniques than you're likely to get by hacking the core.

share|improve this answer
1  
I agree that WP has been fine-tuned for a long time. But I do not think it was fine-tuned to work on crappy hosting, with a particular mixture of plugins. I am curious to see how far can I push it. Even if the changes do not make it into the core, it is good to have a documented way of hacking the core if you think you must. –  Roman Zenka Mar 3 '11 at 2:46
add comment

How can we make Wordpress initialize its environment in memory only once, and then reuse it many times for each hit?

It's called opcode-caching.

http://en.wikipedia.org/wiki/PHP_accelerator

share|improve this answer
1  
I am going to give APC a try and see what happens. When I originally asked that question, I meant more than just opcode caching - I meant reusing the entire environment that WordPress builds - code + data. Memcached will help getting you the data faster, but you will still be cloning the data in the server memory. Now it seems that opcode caching would potentially take care of ~90% of all memory consumption. –  Roman Zenka Feb 28 '11 at 0:58
    
If you have the resources for some experiments, you might also try setting up a FastCGI environment. I'd be very interested in some comparisons between mod_php and running under FastCGI. –  Dougal Campbell Feb 28 '11 at 14:01
add comment

And this is why I think WordPress is in serious need of a rewrite. You can no longer blame its memory consumption on sheer complexity of what it does. It simply does things wrong.

What a naive conclusion. Read Things You Should Never Do, Part I.

Thanks for the memory usage plots, though.

share|improve this answer
    
True, it is naive. Maybe I should have said "refactor" instead of "rewrite", then it sounds much better. Post updated. –  Roman Zenka Feb 27 '11 at 15:29
2  
Ok, well, if you have specific suggestions (especially patches), you're welcome to post them on trac: core.trac.wordpress.org –  scribu Feb 27 '11 at 15:33
    
I am working on it right now. I am trying to plot a map of objects in the memory, so I can see how much is used by what. Is there a tool that would take a memory dump and plot it? –  Roman Zenka Feb 27 '11 at 15:35
5  
@scribu - +1 for the link to Joel's post! –  MikeSchinkel Feb 27 '11 at 17:55
1  
Ok, just keep in mind that WP_Object_Cache can be replaced with a memcached implementation etc. –  scribu Feb 27 '11 at 19:58
show 4 more comments

Starting with WordPress 3.2, PHP 5.2 will be the minimum requirement. I think with that under our belts, bits of the core can start to be restructured, and use classes with auto-loading. This would let us avoid loading some chunks of code unless they were actually needed. For example, if there were no embeds or galleries in a page view, we might be able to avoid loading a lot of the media code.

However, even if they decide to go that route, I would expect it to be a slow evolution (like much of the other under-the-hood changes that have occurred). It would require shuffling around the location of a lot of files and code, which could possibly break backwards compat for some plugins.

Part of the problem (if it can really be called that) is that without that kind of conditional loading, the core framework can't know ahead of time which functionality it will need or not need in order to generate the content view. So a lot of functions have to be loaded up just in case they're needed.

share|improve this answer
    
@Dougal Campbell I started a bounty on this question, to see if we can hack at least this one instance of WordPress bad enough to get at least 30% memory consumption improvement now, relatively painlessly. It could inspire some of the future development. –  Roman Zenka Feb 27 '11 at 23:22
    
Conditional loading, while potentially reducing memory consupmtion, hurts speed when opcode caching is involved. We tend to favor speed. –  scribu Feb 27 '11 at 23:22
    
More thoughts on autoloading: stackoverflow.com/questions/4788452/… –  scribu Feb 27 '11 at 23:29
    
@scribu When you say "conditional loading", are you talking about autoloading, or actually loading code based on a condition? How much it hurts the speed? –  Roman Zenka Feb 28 '11 at 17:26
1  
Thank you! As I said, I don't know if the WP core will ever take that route (the refactoring required might be too extreme). But I was very impressed with the effort you put into analyzing this, and the graphs you produced. Keep up the good work! –  Dougal Campbell Mar 7 '11 at 13:41
show 3 more comments

Yes, WordPress loads up everything first and then does what we ask it to do. I can recall somewhere that we can create a virtual pool in RAM where we can put in files. I had the idea of putting the whole WordPress in memory (<10MB) & then we can save a lot of I/O which alone should give a speed boost. But I never got the chance to try it and moreover I am not really that much proficient in pursuing something like that. But it looks worth a try.

share|improve this answer
    
And I agree with Rarst to use a static cache plugin so that no processing is done at all. But that can be used with good dynamics too. :) –  Ashfame Feb 25 '11 at 19:36
    
I like that idea. I am not sure how much of this problem is due to I/O latencies, and how much is due to PHP slowly chewing on the data. Do you know of a way how to tell? –  Roman Zenka Feb 26 '11 at 14:52
    
Sorry its just an idea in my head. May be it doesn't effect performance that much what it may seems as data is generally read from hard disk as blocks, so lots of other data required might already have been fetched. I am not too sure. –  Ashfame Mar 6 '11 at 9:51
add comment

Not worth the trouble. WordPress doesn't eat a lot of memory just-because. It eats a lot of memory because it runs a lot of functionality under the hood.

It is far more easier and efficient to cache results (page generated) with static cache plugin and serve that. That way most visitor will not even hit WP itself.

share|improve this answer
1  
I am already using a cache, but I still have a few pages that are genuinely dynamic in nature (e.g. a shopping cart). And when the stars are not alligned correctly, the user can wait for 20 seconds - that is, granted, on GoDaddy, but even if not, I think it would be ~3 seconds at least. I simply cannot provide the kind of really snappy experience people are used to from Google. –  Roman Zenka Feb 26 '11 at 14:50
7  
@Roman Zenka if you have specific performance needs then you are better of looking for specific solutions, rather than hoping that WordPress itself will magically become super fast and light on resources. The first things I'd suggest to look at are opcode cache and fragment static caching... But before that you need to benchmark the heck out of it and determine not only where memory is going, but also where time is spent. WordPress is environment, not a bottleneck by itself. Bottleneck is in what you make it do. –  Rarst Feb 26 '11 at 15:20
    
@Rarst I actually did benchmark the CPU usage and I cannot point finger on any specific place that causes the trouble. Similar to memory - it seems to be spread all over the place. However, my benchmarking might not be done in an optimal way - I use the XDebug profiler and Cachegrind - it is for instance quite hard to tease apart latency due to the database calls. I'd be grateful for pointers to better profiling techniques. –  Roman Zenka Feb 26 '11 at 16:45
    
@Rarst Profiling screenshot was added. –  Roman Zenka Feb 26 '11 at 16:54
3  
It could also be GoDaddy's servers being slow. They're known for not having the greatest hardware and would "rather pay for television advertisements than upgrade their servers" –  Zack Mar 1 '11 at 20:04
show 1 more comment

Not the answer you're looking for? Browse other questions tagged or ask your own question.