How can I free up the memory used by update_post_meta?

Question

I've noticed that each call to update_post_meta results in a small increase in my memory usage.

Normally, the increase is small enough to not matter. But I'm writing a CSV import function that calls it 10,000+ times, which eventually results in an out-of-memory error.

Here's a minimal code example:

function update_business($myData)
{
    error_log("Used memory 1: " . memory_get_usage(false));
    error_log("Allocated memory 1: " . memory_get_usage(true));
    update_post_meta('13663', 'business_id', $myData);
    error_log("Used memory 2: " . memory_get_usage(false));
    error_log("Allocated memory 2: " . memory_get_usage(true));
}

This results in:

...

Used memory 1: 17995848
Used memory 2: 17996992

Used memory 1: 17996992
Used memory 2: 18027720

Used memory 1: 18027720
Used memory 2: 18058448

Used memory 1: 18058448
Used memory 2: 18089176

Used memory 1: 18089176
Used memory 2: 18119904

...

Without the update_post_meta statement, the memory usage stays the same throughout the entire loop (as expected).

But with the update_post_meta statement, the memory usage grows bigger and bigger and never gets freed.

I've tried a bunch of different techniques for clearing the memory, including:

• wpdb::flush
• unset
• gc_collect_cycles
• wp_cache_flush
• wp_suspend_cache_addition(true)

But nothing seems to work.

I've looked at tons of search results and although quite a few people have had this problem, there aren't many answers besides "raise your memory limit" or "do it a different way", neither of which seems like the best solution.

Surely there's some way to free up the memory? I assume that either WordPress, PHP, MySQL or something else is caching something that I don't want it to, or that there's a memory leak somewhere, but I can't figure it out.

Answer 1

It turns out that it was due to Query Monitor, a plugin that records info about each query. Every time update_post_meta ran, Query Monitor would store some data about the query, which eventually added up to be more than the server could handle.

Running my example code on a default theme like Twenty Nineteen with no other plugins enabled results in the expected behavior:

...

Used memory 1: 5480528
Used memory 2: 5480528

Used memory 1: 5480528
Used memory 2: 5480528

...

So if you're running into a similar issue, try disabling all of your plugins and switching to a default theme to help narrow down the issue.

update_post_meta doesn't appear to be the problem, at least not by itself.

Answer 2

While it is good that you found the specific problem in your setup, the general answer is that you are doing it wrong.

Every time that you are doing anything which requires many DB operations you are risking running out of memory, out of bandwidth to the DB server, or out of time. It may get very tricky as something that might work on one server will fail on another.

The right way to do any operation which requires handling of Ks of records is to split it into handling only (for example) 100 of them at a time and use an AJAX based process to control the progress of the operation until all the data was processed.

For sure over time PHP and MySQL became faster and hardware became stronger which is why you might get away with using a naive "brute force" approach in your code, but you are walking on the edge especially if your code is not just a "one-off" and will need to be used again.