Sharing Variables between scheduled events

Question

I'm working on a plugin that scans through a website's content, using a dictionary file to match words. Right now when scanning the entire site it schedules a series of functions as events to run immediately so it can run in the background.

Right now each function(There's one for each page content, post content, media files, etc.) is handling loading the dictionary file in but we are trying to find a way to make it so that the file only needs to be loaded into memory once and all the scheduled scan events can use it.

We've tried passing the dictionary file stored in an array as an argument of the function that the event triggers, but it appears to be holding it in the database once for each event and it overloads the server's memory.

I've come across the idea of shared memory extensions in PHP, but we cannot guarantee that we will be able to enable this and would prefer another solution if possible.

Is there any way in WordPress to do this or do we have to load the file into an array once per function?

Answer 1

In a literal sense? No, PHP instances are separate, else we'd be discussing multithreading

The problem you face is more fundamental than WP, the only way you'll be able to keep this in memory is via object caching, which will let you put a copy of the data structure in easy reach.But it won't save memory, accessing the data will create a copy, and if your machine is not disk IO limited then the performance gains will be minimal

Basically, you would use WP Cache to store the data in the object cache, then make sure there's an object cache dropin for Redis/Memcached/etc

Using WP Cache functions without this, will keep the data in memory until the end of the request. Using plugins that use file based caching will give you no benefit, and you'll need to install Redis/Memcached/etc at a server level