2

I'm wondering if there is some sort of equivalent to Drupal's Batch API in WordPress?

Basically, the API allows you to register a batch request and Drupal will call your function over and over again (with Ajax requests) until your batch is finished. So if you need to perform a complicated update on 1,000,000 posts, you can do so in a batch without having PHP timeout or run out of memory. (You just have to tell Drupal how far along you are and it will update the progress bar for the user, etc.)

I need to do something similar in WordPress (perform a huge batch on the backend of WordPress without the command line). How would you accomplish this?

9
  • Why "without the command line"? WP got WP-CLI, which is equivalent to Drush in Drupal. Also in which scenario does the query run? Is it during an import with a custom importer? Have you already read about AJAX in WP?
    – kaiser
    Commented Oct 16, 2014 at 16:08
  • Because the script needs to run on an environment (i.e. WPEngine) that does not offer access to the command line. also your link is a 404. Commented Oct 16, 2014 at 21:10
  • I fixed the link. Do you have a specific environment or is this a plugin that you will use on different servers? Btw, I'd really stay away from WPEngine - read their Twitter threads. About AJAX: If you use promises, you could simply set a batch size, wait until the promise is fulfilled and then call the whole script with the next part of the stack. You will find examples for promises/deferred processing in the link.
    – kaiser
    Commented Oct 16, 2014 at 21:23
  • It's a plugin I'll use on different servers. And that's good to know about WPEngine, but I'm not the one using it. :) And I know about WordPress' Ajax API, but that's pretty much the equivalent of using hook_menu() in Drupal with a type of MENU_CALLBACK, which provides an ajax callback which is what WordPress' Ajax API does, this is not even close to the Batch API, but thanks for the help. Commented Oct 16, 2014 at 21:33
  • @kaiser and for you reference, the Twitter reviews of WPEngine are not terrible. Commented Oct 16, 2014 at 21:51

3 Answers 3

2

No, there isn't such an API, but if you'd like one you can:

  • Build it yourself
  • Open a ticket on WP Core Trac

But putting that aside, if you need to delete, create, or update 1 million posts or something of a similar scale, doing it via the frontend is not a good thing, even with an AJAX powered progress bar.

3rd party plugins have indeed implemented things similar to this, but not in a super generic way. There is no 'generic batch ajax request' plugin. An example would be the thumbnail regenerator plugins.

In the future, it may be better to use the upcoming REST API to do your requests. You can then make your javascript requests and change posts in batches or 1 by 1, and implement your progress bar.

As a developer who's worked with Managed WordPress hosting

I doubt WP Engine would be happy with this, nor would your client ( 1 million requests to update posts, 1 million hits, that's a lot of hits on a service that uses hits to determine pricing ). So what you propose is potentially going to be quite expensive in $$$ terms, $1 per 1000 hits on overage pricing, I doubt your client will be pleased

So unless you're on the WP Engine enterprise package with millions of hits, you could be:

  • Business! 400k hits, 600,000/1000 = $600 for the first million deleted + $1k for every additional million, + traffic/1000
  • Professional! 100k hits, 900,000/1000 = $900 for the first million deleted + $1k for every additional million, + traffic/1000
  • Personal! 25k hits, 975,000/1000 = $900 for the first million deleted + $1k for every additional million, + traffic/1000

It may cost more or less on other managed WordPress services, but it's still a risky proposition, especially if it doesn't work right the first time and you need to repeat the process

But even if you ignore pricing, those are a lot of requests, and requests take time. Even if they take a tiny amount of time, that adds up. Your progress bar is going to be waiting for a while...

What I would do

Instead it would be better to pull down the site locally, run your commands using WP CLI, then upload the new database. You have a git repo with your code, you have SFTP access, you have PHPMyAdmin for the database, it can be done. Even the majority of $1pm shared hosts provide these facilities.

You don't need SSH access when you can pull it down into an environment totally under your control, and the download and upload will be a lot faster than hundreds of thousands of AJAX requests.

As a bonus you get a free up to date copy of the live content!

2
2

I have no idea why would you even want to do something like that. Lets assume that a ping time to the server is 100ms (mine is usually worse), this means that with simple AJAX that sends, wait for response and then sends again, 1M requests will take 100k seconds - more then a day. if you can send 10 at each time you will cut it down to about 3 hours but you will basically doing a DOS against the site.

The only point I can see with doing something like that in AJAX is to have a human watch what is going on and handle unexpected events, but when the process takes more then an hour no one will want to use this kind of method.

The right way to handle such a need is by using the wordpress cron. Do one batch and immediately schedule an event to run the next one. That will be much faster without any unreasonable demands of php execution or memory if you wrote the process right.

2

Actually starting since WordPress 5.6, there is such an API. Check out this blog post from November 2020 talking about the REST API Batch Framework.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.