Is it safe to run wp-cron.php twice if the first instance takes too long?

Question

Let's say I disabled "DISABLE_WP_CRON" in wp-config.php

Is it OK to run wp-cron.php twice overlapping?

I've noticed sometimes wp-cron.php needs several minutes. I'd give it a full hour but the blogger I work with does a lot of scheduling through the day, she's very busy.

Like say I have it running every 10 mins with a real crontab

*/10 * * * etc.

And it needs a full 15 minutes to do its business. 15 minutes is just a random example here. There doesn't seem to be any rhyme or reason for how long it takes.

Can two "instances" of wp-cron.php safely run together? Does the oldest instance die and the new one takes over? From what I can tell from my bash script that runs wp-cron.php, the first instance fails when the new instance starts.

Example: cron kicks off wp-cron.php at 11:01pm.

php /home/www-data/example.com/wp-cron.php && echo "wp-cron.php done" | mailx [email protected]

11:21pm wp-cron.php starts again (*/20) and I still haven't got my "done" email. But this time I get the "done" email 17 seconds after the subsequent wp-cron.php ran.

(Not sure if wp-cron.php failed on its own or if wp-cron.php killed itself after running 2x.)

So I'm wondering: Is data lost from the first instance? The abrupt failure of a process that was running for 10 minutes seems "bad" to me. At this point, I'm tempted to stop using crontab and go back to the wp-config default behavior because this is way more trouble than I bargained for.

This doesn't sound very reassuring either: How is WP Cron Locking implemented?

What if the "transient" is not deleted before the first process dies? Are the unfinished tasks forever unfinished then? And what is a transient? I assume it's some kind of flag.

I'm having major problems with this today. Server got slow, triggered some timeouts which I assume caused wp-cron.php to fail, resulted in double postings, scheduled posts not posting, what a nightmare!

Update: Attempting to call wp-cron.php from the operating system with crontab was a huge waste of time for me and the client. On my end, wp-cron.php seemed to execute fine, used memory and CPU time for various durations, but the scheduled posts never posted according to my client. It would be nice if there was a "verbose" mode to indicate what it's actually doing. I ran out of time to investigate this and went down another route. I just worked on improving the speed of the server which somehow fixed the problem. (My guess is, some timeout was reached because the server was overloaded.) I was hoping I could just get a crontab working quickly before I worked on upgrading the server. But the more I looked at wp-cron.php, the more I distrusted it. Failed scheduling, double posts, conflicting advice how it should be executed. So that's why I came here. I have a similar script for one of my own websites that runs hourly--it's not designed to overlap itself but it's fairly safe if it does, within reasonable parameters. Then again, I know what's reasonable to me. I don't know what's reasonable to the author of wp-cron.php.

So this was a distraction and a dead-end for me. I changed wp-config.php back to the default (deleted DISABLE_WP_CRON) and let WordPress deal with wp-cron.php. Now with a faster server, WordPress schedules posts again just fine. Always safer to stick to defaults.

More details of the story here: http://tomakefast.com/when-wp-cron-php-sceduling-fails-3003/

Answer 1

How wp-cron.php behaves depends on how you execute it. If you execute it by HTTP request with 'doing_wp_cron' in the GET string, it will check if another wp-cron process has set a lock and exit if it has. By default, this is how WordPress executes it. It is easily done in crontab like so:

*/10 * * * * /usr/bin/wget -q -O "http://www.example.com/wp-cron.php?doing_wp_cron=`date +\%s.\%N`" > /dev/null 2>&1

But if you execute it directly with the PHP cli, as you are doing, then it's different. Then it checks WP_CRON_LOCK_TIMEOUT, which by default is set to sixty seconds. If the existing process has been running for longer than that, it claims the lock for its own and proceeds to run scheduled jobs starting all over at the beginning.

This is not a disaster because wp-cron reschedules and unschedules jobs as it goes. Right before it runs a job, it unschedules it. This prevents the job from running twice[1]. And right after it finishes executing a job, it checks to make sure it still owns the lock. If it does not, it exits. Which is why you are seeing the first process quit shortly after you start the second.

So in your case, you have a set of scheduled jobs that sometimes take more than twenty minutes to run. Probably there are some jobs in the every10minutes queue that are taking up all that time. wp-cron starts and chugs away for twenty minutes. Then another instance starts. That second instance sees that the lock is older than the sixty second timeout and claims the lock for itself. It then starts over at the beginning. The first process unscheduled and rescheduled jobs as it went, but if the jobs are in the every10minutes queue, they are now ripe to be run again. So the second process starts them over again. The first one gets to the end of the job it was on, sees it no longer owns the lock, and quits. And because the scheduled posts are at the end of the queue, they never get posted.

Also, because wp-cron unschedules jobs just before running them, if it were to die between doing that and the job completing, due to memory running out or something, the job would just be lost.

1. Most of the time. There is a race condition. If another process both a) claims the lock after the first process checks it and before that process unschedules the next job, that job will be run twice. On a bogged down server, the likelihood of hitting that condition might be larger.