What's the case against transient-ing almost everything that's mostly static?

Question

Assume I have a menu that's simply static, for most sites, or rather, let's use my example, a blogging site, it almost never, never changes.

Why not hook it to a transient that never expires unless changed? So, if you play with that menu, you'd reset the transient and get the newer instance, but otherwise, it'll never be retrieved with the full query itself.

Same for comments, same for, well, almost everything.

Is this a bad idea?

I understand this is more philosophy-architecture oriented but I think it's an important question with deep implications.

Answer 1

Actually, doing it for menu is good, you can even store entire HTML output and cut not only queries, but processing time too. Additionally, you can use WP_Object_Cache directly in conjunction with Redis/Memcached backend and avoid even single DB query for the transient itself.

Re comments - they do change, so running WP_Comment_Query is unavoidable. However, you can cache them as well, for some short amount of time. Say, 5 minutes. In this case, you need to find the balance between how much speed boost you gain and the delay before users will see their newly made comments.

Rule of thumb is, cache when needed, when it gives you positive impact and does not introduce negative impact.

WordPress already does great job in caching most important things (including menu items) - just put the object cache class in WP-content and add object cache backend on your server.

Answer 2

After a few months of doing this, I believe that my original question is very well founded and while this technique is surely not new, here's what I'm doing to greatly optimize a lot of things in my system:

(It's more philosophical, but I'll provide code)

I'm not going to discuss the, hopefully, understood fact that you shouldn't transient items / queries that are going to impact your work-flow, nor should you transient everything, because that overloads the database, especially with a lot of big queries - an example would be, on a not-so-good host, transienting the posts of each user (assuming you have a big community) in throves is a big no-no. You might have inactive users that have 10-15 posts each, but no one goes to their page, so their data will just stay idle in the database, just waiting for someone to access it to reset. (This is solved by just creating the transient on that user's page, which, hopefully, you're already doing!)

Regardless, that's exactly how we must do it: Instead of thinking of transients in terms of expire-on-time, we should approach them from a expire-on-trigger way.

So, instead of:

set_transient( 'name', 'value', 3600 );

We should simply set the transient without a time and let a "cleaner object" handle it. Enter architectural choices. We must look at what exactly that transient saves / in what manner and decide when exactly it is best to delete it. So, let's assume we're dealing with an user's posts and that we won't re-update his transient on a timer. What we can do is hook to save_post and just re-create that transient every time the user posts a post. This way, our transient will only ever update itself when the user himself is creating a new post:

add_action( 'save_post', function( $post_id, $post, $update ) {
    $author = get_post_field( 'post_author', $post_id );
    /**
     * But if he is, then we'll delete his posts' transient.
     */
    delete_transient( 'user_posts_' . $author->ID );
});

What we achieved here is never deleting a transient, unless it needs to be deleted.

Now, that transient will be re-created when somebody hits that user's page and not be on a timer.

The only problem is that the database now grows larger, so we need a garbage collection mechanism that doesn't keep transients that aren't "popular" in the database, so we don't clutter it.

I've successfully applied this to transients related to the media library images, where I would need a transient, but I'd also like a trigger to update it.

Finishing this, the more I look at this, the more I think it should be called "knowing when you need to update a transient".

So far, it has solved way more issues for me than it caused but I agree that it's uncertain whether or not this would scale well. At the same time the issue of needing to scale is a great issue to have and that, if you code the system this way, you only need to disconnect these deletion mechanisms for it to work in the traditional way.

So, exactly what has this helped me achieve? What are the benefits?

Transients without actually feeling like they're transients. Now, whenever you do an action, you see its result, where-as with the normal way, you'd need to wait for a time to see the effect (after the transient expired). Just refresh and the new data is there, no need to wait, no need to lose on the benefits.

In fact, it somehow makes more sense to delete transients on a certain trigger rather than on a timer.