Last-Modified header support doesn’t speed up server processing - want more

Question

I’ve just added “Last-Modified” / “If-Modified-Since” header support for some pages on my site via the recommendation here, but I didn’t see a very noticeable client page load speed improvement. This makes sense as I think about it, as follows:

The server-side page build process still occurs, but depending on the client request header's “Last-Modified” value (and others), the page content built by the server won’t be returned if it’s not needed by the client. This would be very good for large payloads, but it saves negligible time for my users for smaller payload pages. (Yes there can be other benefits too, but that's not my focus here.)

Has anyone taken the next optimization step in this direction for Wordpress? If the request “If-Modified-Since” request header is prior to the “Last-Modified” value, then short circuit the page build process, and return the header only?

Succinctly: Is there a way to short circuit the Wordpress page content generation process, and return the response header only?

… or … does anyone know a different/better next optimization step?

From my partial understanding of WP, it would seem that you would need a hook to intercept/replace the page template processing, and that the primary 'gotcha' would be if (other) plug-ins that could be interrupted also wanted to manipulate the response headers.

Answer 1

You need to have a lot of traffic for naive protocol based optimizations to be effective, and even then it is not obvious that you can gain much even when it does kick in.

Most people will be first time users of your site, and browser have only a limited space to store information about the last update time of all the pages a user visited.

But even if you actually get such a request, how exactly do you know when was the last update? The best answer to that question that wordpress has is the publish time of the latest published post, which might not represent at all when a theme change, or any other setting change happened. But even for such a naive assumption, you still need to load wordpress and do at least two DB queries. You will probably get some improvement, but not as much as using an object cache or page cache in the first place.

And in the real world, most of the slowness is because of the JS/CSS/media on the page, and if it is not cached (and unless you are facebook, it is unlikely t be cached), your page will still load slow even if the "last published" thing did work.

Answer 2

I'm answering my own question. Perhaps it will be useful for someone else.

Short answer: You can short-circuit the page generation process via wp_die() or wp_send_json().

But since I don't want to send a 500 status it eliminates wp_die(), and since I want to send html not json, it eliminates wp_send_json(). So instead, I copied & modified the wp_send_json() source to produce the following:

function wp_send_html( $response = null, $status_code = null ) {
    @header( 'Content-Type: text/html; charset=' . get_option( 'blog_charset' ) );
    if ( null !== $status_code ) {
        status_header( $status_code );
    }
    if ($response) {
        echo $response; 
    }
    die;
};

In my code, after I determine the request is unchanged from the prior request, I call:

wp_send_html(null, 304);

... which sends a 304 response including any html response headers that have setup, and an empty body.

Result: ~75% page load time savings

In the single post places that I'm using this, I've decreased response times from about 4-5 secs, down to 1.0-1.3 seconds.

Longer answer:

From @Mark's post, the primary things I need to consider are the post last change, and the last change for any theme or other global element. I am only handling only single page/post pages, and I've added a $MIN_UPD_DATE global at the top of my functions.php with I'll update when I do global/style changes (yeah, I know that's a little kludgey).

My more complete code in my functions.php is as follows:

// Update this on sitewide changes
$MIN_UPD_DATE = DateTime::createFromFormat('M d Y H:i:s', 'Sep 01 2018 01:01:01');

// For web pages and single post pages - note the last changed date
    // Thanks to: https://wordpress.stackexchange.com/questions/172966/if-modified-since-http-header
    // This will support clients sending HTTP header: If-Modified-Since 
function handle_modified_since_header() {
    global $MIN_UPD_DATE;
    //Check if we are in a single post of any type (archive pages have no modified date)

    if( is_singular() ) { // excludes multi-post pages
        $post_id = get_queried_object_id();
        if( $post_id ) {
            header("Cache-Control: public");  
            // inherited/default was: Cache-Control: no-cache, must-revalidate, max-age=0

            $postModTime = new DateTime(get_the_modified_time('D, d M Y H:i:s', $post_id));
            if ($MIN_UPD_DATE > $postModTime) {
                $postModTime = $MIN_UPD_DATE;
            };
            header("Last-Modified: " . $postModTime->format("D, d M Y H:i:s") . " GMT" );
            return true;    // can use modified date to expire page
        }
    }
    return false;   // can NOT use modified date to expire page
};

// Checks single post & page entries for a Post Modified Date 
//  after the http request header: IF_MODIFIED_SINCE
//     or after the MIN_UPD_DATE
//  else, on any missing elements, assumes that request is expired.
function is_request_expired() {
    global $MIN_UPD_DATE;
    $MOD_SINCE   = 'HTTP_IF_MODIFIED_SINCE';
    $DATE_FMT    = 'D, d M Y H:i:s O';
    $postModTime = null;
    $isExpired   = true;
    $httpLastUpdate    = null;

    if (!isset( $_SERVER[$MOD_SINCE] )) {  // Quit on no http last mod date
        return $isExpired;
    };

    $httpLastUpdate = DateTime::createFromFormat($DATE_FMT, $_SERVER[$MOD_SINCE]);
    if (!$httpLastUpdate) {       // Quit on can't decode last mod date
        return $isExpired;
    }

    $post_id = get_queried_object_id(); // Get Post last modified date
    if( $post_id ) {
        $postModTime = new DateTime(get_the_modified_time('D, d M Y H:i:s', $post_id));
    };

    if ($postModTime) {
        // http last-mod-date is before post-list-mod-date or before min-upd-date
        $isExpired = ($httpLastUpdate < $postModTime) || ($httpLastUpdate < $MIN_UPD_DATE);
    };
    return $isExpired;
}

function send_on_not_expired_single( $wp_query ) {      
    if ($_SERVER['REQUEST_METHOD']!=='GET') {  // ignore for non-GET requests
        return $wp_query;
    }

    $use_modified_since_rule = handle_modified_since_header(); // conditionally set Last-Modified http header
    if ($use_modified_since_rule) {
        $is_expired = is_request_expired();
        if (!$is_expired) {
            wp_send_html(null, 304);
        };
    };
    return $wp_query;
}
add_filter( 'parse_query', 'send_on_not_expired_single', 200);

/**
 * Send an HTML response back to an html request.
 *              patterned on the v4.9.8 WP codex wp_save_json
 *              from: https://core.trac.wordpress.org/browser/tags/4.9.8/src/wp-includes/functions.php#L3179
 *
 * @param mixed $response    Variable (usually an array or object),
 *                           then print and die.
 * @param int   $status_code The HTTP status code to output.
 */
function wp_send_html( $response = null, $status_code = null ) {
    @header( 'Content-Type: text/html; charset=' . get_option( 'blog_charset' ) );
    if ( null !== $status_code ) {
        status_header( $status_code );
    }
    if ($response) {
        echo wp_json_encode( $response );   
    }
    die;
};

As a final note, per some comments/suggestions, I did review some caching plug-in's. Although some of them do have very good 'last-modifed-date' checking/updating logic (which I don't) for when the site presentation is updated, the ones that I reviewed either specifically exclude logged in users from caching, or have significant limitations for logged in users which eliminate their usefulness for my focus.