Update Content based on expiration time or queued processing

Question

every hour by cron, I call a function that updates the contents. If on post have custom field with auto,so work.(to distinguish it from those written manually).

 function aggiorna_dati() {
            $args = array(
                'post_type' => 'post',
                'posts_per_page' => -1 ,
                'post_status'    => 'private'
            );
        
            $articoli = new WP_Query($args);
            if ($articoli->have_posts()):
                while($articoli->have_posts()): $articoli->the_post();    
        $auto = get_post_meta( get_the_ID(), 'auto', true);
        if  ($auto) {
                    funzione_recensione($articoli);
            
        }
        
                endwhile;
            endif;
    
    }

the function are callback by cron with this code:

function dg_cron_schedule_aggiorna_dati() {
    $timestamp = wp_next_scheduled( 'dg_aggiorna_dati' );
    if ( $timestamp == false ) {
        wp_schedule_event( time(), 'hourly', 'dg_aggiorna_dati' );
    }
}
add_action( 'init', 'dg_cron_schedule_aggiorna_dati' );

The problem is that function update all article same time, but this is not possible because the function that generates contents (funzione_recensioni) works with external api,which are limited in number and call / hour frequency. herefore, I would need to process the queue slowly, for example, four articles every hour, or by setting an expiration date, before regenerating the page, (for example once a day)

thanks to those who can help me

Answer 1

Few things come to my mind, which could help.

Post status

Perhaps you could use the pending post status, which you would use as a parameter in aggiorna_dati(), for the auto-generated posts that needs to be processed by funzione_recensione(). After the function is done it would change the status to something else.

$args['post_status'] = 'pending';

Posts per page

Instead of using some counter inside the loop, you could set the posts_per_page parameter to the max number the API can handle per hour. This would however require you to either track the paged parameter somehow (transient, custom option, hour of the day...), so that you don't just loop the first n posts over and over again, or marking the processed posts, one way or the other, so that they will be excluded next time the query runs.

$api_hourly_max_capacity = 4;
$args['posts_per_page'] = $api_hourly_max_capacity;

Date query

If the auto-generated posts don't need to be updated every hour, then you could limit the query with date parameter and for example only get posts that haven't been updated today. For this to work, funzione_recensione() should update the processed post, when the processing is done, so that its' modified date gets updated.

$args['date_query'] = array(
    array(
        'column' => 'post_modified_gmt',
        'before' => 'today',
    ),
);

Meta query

As a side note you can limit the posts query with meta parameters. This way you don't need to have the if statement inside the loop as the queried posts are already matched. If using the pending post status is not an option, you could consider adding a helper meta value to track which posts need to be processed to further limit the posts query.

$args['meta_query'] = array(
    array(
        'key'     => 'auto',
        'value'   => true, // or whatever the value is when is auto-generated
    ),
    // helper meta maybe?
    array(
        'key'     => 'process_state',
        'value'   => 'pending',
    ),
);

Although using a custom private taxonomy to track these kinds of things (is auto-generated, needs processing) would be a better choice performance-wise.

Regarding the loop

If you don't have a strict need for the WP template functions, the you can skip setting up the loop with have_posts() and the_post() as you can access the post data directly via the post object properties and loop found posts with a PHP native function.

$post_id = $post->ID;
array_walk( $articoli->posts, 'funzione_recensione' );

P.s.

As another side note, you can save a little processing time and make the query perform a bit better by adding extra limits to it.

$args['no_found_rows'] = true; // if pagination is not needed
$args['update_post_term_cache'] = true; // if terms are not needed
$args['fields'] = 'ids'; // if only ids are needed

EDIT: simplified code example of the above explanation.

// optional helper taxonomy
add_action(
    'init',
    function() {
        // minimal example
        register_taxonomy('my_private_tax', 'post', array(
            'public' => false,
        ));
    }
);

function some_function_to_auto_generate_posts() {   
    // generate the post
    $post_id = wp_insert_post(array(
        // title, content, etc. for the post..
        // ...
        // upon creation, set the status pending to signify the post is waiting for content
        'post_status' => 'pending',
    ));
    // append private taxonomy term to the new post
    // you could 
    wp_set_object_terms( $post_id, 'is-autogenerated', 'my_private_tax', true );
    // OR if taxonomy is not used, set post meta
    // add_post_meta( $post_id, 'auto', 'true' );
}

function aggiorna_dati() {

    $api_hourly_max_capacity = 4; // or what ever the actual limit is

    // OPTION 1, if the posts are processed only once
    $posts_waiting_to_be_processed = 'pending'; // if using this post status is an option
    $args = array(
    'post_type' => 'post',
    'posts_per_page' => $api_hourly_max_capacity,
    'post_status'    => $posts_waiting_to_be_processed,
        'no_found_rows' => true, // pagination is not needed
    );

    // OPTION 2, if the posts are updated daily
    // $posts_waiting_to_be_processed = array('pending', 'publish');
    // $args = array(
    //  'post_type' => 'post',
    //  'posts_per_page' => $api_hourly_max_capacity,
    //  'post_status'    => $posts_waiting_to_be_processed,
    //  'no_found_rows' => true, // pagination is not needed
    //  'date_query' => array(
    //      array(
    //       'column' => 'post_modified_gmt',
    //       'before' => 'today',
    //     ),
    //  ),
    // );

    // if the private taxonomy is used (better performance)
    $args['tax_query'] = array(
        array(
            'taxonomy' => 'my_private_tax',
      'field'    => 'slug',
      'terms'    => 'is-autogenerated',
        )
    );
    // OR do the meta query, if the custom taxonomy is not used
    // $args['meta_query'] = array(
  //   array(
  //     'key'     => 'auto',
  //     'value'   => 'true',
  //   ),
    // );

    // query posts matching the $args
    $articoli = new WP_Query($args);

    // OPTION A, loop found posts
    array_walk( $articoli->posts, 'funzione_recensione' );
    // this is the same as doing
    // foreach ($articoli->posts as $post) {
    //  funzione_recensione($post);
    // }
    // OPTION B, pass the query to the processing function
    // funzione_recensione($articoli);
}

// OPTION A, pass one post at a time to the processor function
function funzione_recensione( WP_Post $post ) {

    // fetch data from API, do some processing, etc.

    $post_status_after_processing = 'publish';
    $processed_post_data = array(
        'ID' => $post,
        'post_status' => $post_status_after_processing,
        // other post fields that get updated...
    );
    
    wp_update_post($processed_post_data);
}

// OPTION B, pass the whole query to the processor
// function funzione_recensione( WP_Query $query ) {
//  // fetch data from API, do some processing, update posts, etc.
// }