0

So I wanted to see if someone might be able to help me with this, so below I will explain what I've done:

  1. I am using wp_remote_get to grab the posts from another website with the arguments page and per_page.

  2. Next, I am checking to see if the post has already been imported using $current_post_id = post_exists($post->title->rendered); and if it does, don't call the wp_insert_post function.

Here is what I have working already:

If I call the function, it will check those 100 posts on page 1 and check the titles and only import posts that are not matching.

What I'm trying to achieve:

  1. Let's say that I have 402 posts, how do I dynamically loop through all the pages and then when it gets to the last page, reset back to the page 1?

Here is the code:

public function get_posts_via_rest_api(): void
{
    if (!is_admin()) {
        require_once(ABSPATH . 'wp-admin/includes/post.php');
    }
    $response = wp_remote_get(
        add_query_arg( [
            'page' => 1,
            'per_page' => 100
        ],
            'https://www.website-posts-live-at.com/wp-json/wp/v2/posts')
    );
    if (is_wp_error($response)) {
        return;
    }
    if ($response['response']['code'] === 200) {
        try {
            $posts = json_decode(
                $response['body'],
                false,
                512,
                JSON_THROW_ON_ERROR
            );
            if (empty($posts)) {
                return;
            }
        } catch (Exception) {
            error_log(
                print_r(json_last_error(), true)
            );
            return;
        }
        foreach ($posts as $post) {
            $current_post_id = post_exists($post->title->rendered);
            if ($current_post_id === 0) {
                $my_post = [
                    'post_type'     => 'post',
                    'post_status'   => 'pending',
                    'post_title'    => wp_strip_all_tags($post->title->rendered),
                    'post_content'  => wp_strip_all_tags($post->content->rendered),
                    'post_excerpt'  => wp_strip_all_tags($post->excerpt->rendered),
                    'post_author'   => 1
                ];
                $post_id = wp_insert_post($my_post);
                wp_set_object_terms($post_id, 'Global', 'category');
                wp_set_object_terms($post_id, 'Global', 'post_tag');
            }
        }
    }
}
6
  • are you asking how to find out the number of pages? This is covered in the official REST API handbook developer.wordpress.org/rest-api/using-the-rest-api/pagination, there are HTTP headers in the response that tell you which page and how many there are. You may also find this will not scale particularly well unless you're running this code via CLI, post creation has a cost, and you have a time limit for running this code, the REST API limits itself to 100 posts for this reason, but fetching a post is cheaper/faster than inserting a post, you will need to async/batch this process up
    – Tom J Nowell
    Mar 20 at 0:59
  • @TomJNowell, if you need to bring in posts from another site to a new site, what process would you take? I basically need to keep tabs on our global site that releases new content and then pull in content to our sub sites. I don't want to use any plugins, as this will be a custom solution. Corporate want to pull in all the posts with the tag as "Global" and have them actually be posts, where I know is only possible with wp_insert_post.
    – Test2
    Mar 20 at 1:29
  • I'm not suggesting you avoid wp_insert_post, I'm saying that what you're trying to do is too big to do in a single request. It's like a child trying to eat an entire cake in a single bite, the answer is not to avoid biting, it's to break the cake into multiple smaller pieces that can be eaten separately across a longer period of time. It is highly unlikely you will be able to safely and reliably create 100 posts in a single request, especially when you need to make a HTTP request too. Request fewer posts and don't do it all at the same time, fetching multiple pages at once is unrealistic
    – Tom J Nowell
    Mar 20 at 1:34
  • @TomJNowell, thanks for that! I can totally break it down into smaller posts and get like 25 at a time, that's no biggie at all, I can just have a cron run every 5 mins or so, I just need to figure out some logic to loop through the X-WP-TotalPages which you mentioned, thanks for that! I didn't know about those headers. Would you approach the looping for X-WP-TotalPages in any special way?
    – Test2
    Mar 20 at 1:36
  • I'm able to call $headers = wp_remote_retrieve_headers($response); and I see ["x-wp-totalpages"] => string(3) "798", phew it's more than I imagined.
    – Test2
    Mar 20 at 1:40

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.