Async Loading of Custom Posts

Question

I have a wordpress page template, where I'm loading custom posts.

The code for fetching these custom posts looks as follows:

template-parts/content-page.php

<article id="songs">
    <?php get_template_part( 'partials/content/custom', 'songs' ); ?>
</article>

partials/content/custom/songs.php

$args = array(
    'posts_per_page'   => 0,
    'offset'           => 0,
    'category'         => '',
    'category_name'    => '',
    'orderby'          => $orderBy,
    'order'            => $order,
    'include'          => '',
    'meta_key'         => '',
    'meta_value'       => '',
    'post_type'        => 'custom_song',
    'post_mime_type'   => '',
    'post_parent'      => '',
    'author'           => '',
    'post_status'      => 'publish',
    'suppress_filters' => true
);
$songs_array = get_posts( $args );

if (count($songs_array) > 0){
    ?>
    <ul>
    <?php
    foreach ($songs_array as $mysong){
        set_query_var( 'mysong', $mysong);
        get_template_part( 'partials/content/custom', 'normal' );
    }
    ?>
    </ul>
    <?php
}
?>

The problem is that there are over 2000 records. And I want all of them to be loaded at once without any pagination. The above code works and it does load all the posts, but the page is slow because of this query.

Can you please help me how I can optimize this and make the load faster? Is there a way I can load this asynchronously? So that I can show a loading icon in this part of the page till the posts are loaded?

Answer 1

Testing is the only way to be sure, but my guess would be that it is not the query itself that is slowing the page load. After you have fetched those 2000 records, you are looping through them. Every time you call get_template_part. This function in turn calls locate_template, which leads to load_template, where you find the require function that actually loads the template file. There probably is some caching going on, but in theory you are doing 2000 file load requests.

So, I would get rid of that separate template file and integrate it in the loop. It will at least save you several function calls.

Answer 2

Regardless if you are loading through AJAX or not, if you need to load 2000+ records on one page, at one single time or in one single request you are always going to be at the mercy of your server resources to do so.

I would look at caching solutions first as that will be the simplest to set up quickly - then in the meantime query optimisation will be where you save time. Only getting the bare essential data you need to display will help a great deal. For example if you are only showing a link then only lookup the Title & Permalink. This could be done either via WP_Query and the fields parameter here - or a custom MySQL query.

If you want to go down the AJAX route then you will need to make sure that your script only outputs useful data - the partials/content/custom/songs.php template could have it's own url endpoint and you can then call it on page using jQuery - AJAX load() Method. Don't load the whole page and it's associated stylesheets and JS scripts and other resources - headers footers etc and then pick out the relevant DIV to load. Make that page a bare html page with just the data you need.

I seem to also remember something called "chunking" that Nginx or Apache does (can't remember which). I since can't find much info on it either. From memory this returns part of the page to the end user before the end of script has finished loading. This could be worth looking at - if you want the user to see 'something' before the full page is processed / loaded.

Finally - education - tell the client that if your server is slow when you have only 2000 records and they hope to grow their song catalogue you're heading in one direction. To an expensive server / CPU bill. It's usually cheaper to pay a developer to fix / optimise something once vs. paying monthly for higher usage and bigger server bills every month for 2+ years. If you get them to understand this one though please teach me how...

AJAX lazy loading via query pagination is the sensible approach here you could even trigger the 1st lazy load on completion of the 1st page load and put the scroll offset trigger high up - you would probably never see the end of the list before the next bit is loaded....

Sorry - lot of text and not much code today... but be thankful at least. This started as a comment and no one wants to read a comment that long

Answer 3

In your case you can use ajax on page scroll ( Infinite Scroll ) to load more post. On first page load it will load only few ( 20 for example ) post, and when user scroll to bottom and scroll reach to bottom ajax will load more ( 20 ) post. In this way you can asynchronously load post with loading icon.

For infinite scroll there are many plugin are availabe if you want to use. Or you can use own custom code for this, following are links which impliemtns custom infinite scroll, Hope it will help.

1. https://dev.to/erhankilic/adding-infinite-scroll-without-plugin-to-wordpress-theme-595a
2. https://scripthere.com/simple-infinite-scroll-for-wordpress-sites-without-plugin/

Answer 4

A good practice would be to use cron(since it runs in the background https://github.com/deliciousbrains/wp-background-processing) to generate a heavy request (if the request is very heavy, you can add more queues). The bottom line is that cron generates an array of data and stores it in" transient " (you can use the entire layout), and then at the right time you can get this data with a light query to the database.