36

I've been modifying the built in WP search using the pre_get_posts filter, allowing the user to sort the posts (including a bunch of custom post types) by different fields.

The problem I'm having though is that when I tell WP to sort by a meta value it will exclude all posts that don't have that meta value set. This causes the number of results to change if you change sorting from say "Price" to "Date" because "Posts" don't have "Price" set but "Items" do.

This is not what I want, so I'd like to know if there's a way to include ALL posts - even those that lack the meta value I'm sorting on - and put the one's without the value last.

I know how to sort on more than one field but that doesn't help.

Thanks

Seems I'm not the only one with this question: Way to include posts both with & without certain meta_key in args for wp_query? but there's no solution there.

Update

I've tried the answer but not sure if I understood correctly, here's what I have right now:

<?php
function my_stuff ($qry) {
    $qry->set('meta_query', array(array(
        'key' => 'item_price', 
        'value' => '', 
        'compare' => 'NOT EXISTS'
    )));

    $qry->set('orderby', 'meta_value date'); # Sorting works with meta_value as well as meta_value_num - I've tried both
    $qry->set('order', 'ASC DESC');
    $qry->set('meta_key', 'item_price');
}

The meta value is a number (it is used to store a price as the name suggests)

Update 2

I've commented out the order-stuff and all I have now is this:

<?php
$qry->set('meta_query', array(array(
    'key' => 'item_price', 
    'value' => '', 
    'compare' => 'NOT EXISTS'
)));

With this code the query seems to return all posts that don't have the item_price key and none of the posts that have it. I.E. the problem is now reversed.

If I add in the order-code as well I get 0 results.

Edit: ...three years later... :P I had this issue again. I tried all the answers given and none work. Not sure why some people seem to think they work but they don't work for me at least.

The solution I ended up with is using the save_post filter - making sure all posts have the custom field I wish to sort on. It's a bit annoying I have to do it, but at as long as you do it early on you'll likely have no issues.

In this case I was building a "view counter" on posts and wanted users to be able to sort on the most read posts. Again, posts that have never been viewed (I guess that's pretty unlikely - but still) disappeared when sorting on the view count. I added this bit of code to make sure all posts have a view count:

add_action('save_post', function ($postId) {
    add_post_meta($postId, '_sleek_view_count', 0, true);
});
  • Please show us your code. Makes it easier to answer. – kaiser Jun 9 '13 at 23:24
  • First: meta_query and tax_query are always an array( array() ) as they combine multiple arrays. Second - as mentioned in my answer - you need to use meta_value_num for numbers. It might as well be needed to actually define the meta_value_num (see WP_Query-Codex page entry). Last, it doesn't make sense to order in ASC and DESC direction. That's not possible. The space delimiter only works for orderby and you can't tell it to sort the first ASC and the second DESC. That's what posts_clauses filter is for. – kaiser Jun 10 '13 at 0:42
  • And make sure that your meta_value_num entries are real numbers. Seen it too often that someone states that it's a number, but in fact saving it as a string in the database. – kaiser Jun 10 '13 at 0:43
  • Thanks for your help, I will try this and get back to you. The reason for ASC DESC is so that it sorts on the meta_value in ASC and the date in DESC, as far as I can tell it works. – powerbuoy Jun 10 '13 at 1:06
  • 1
    @Howdy_McGee that is correct. Some of my custom post types HAVE this value set. Some don't. And the built in post types (like POST and PAGE) do not. So whenever I try to sort on that custom field only posts WITH the custom field show up. – powerbuoy Apr 15 '14 at 18:31

11 Answers 11

4

There's two possible solutions to this:

1. All posts have meta

The best solution I have found here is to give the rest of the posts/products an item price of 0. You can do this manually, or loop through all the posts and if the price is empty then update it.

To make this manageable in the future you can hook into save_post and give them a value when they are first added (only if it's blank).

2. Multiple Queries

You could run the first query as you're doing and store the IDs of the posts returned. You could then run another query for all posts and orderby date, excluding the IDs return from the first query.

You can then print out the two results separately order and you'll get the desired results.

  • 1
    Three years later and I had the same issue again :P Had to use the save_post method (I've updated my question with the code I used). – powerbuoy Jun 7 '17 at 9:11
8

This method will return all of the posts including those with and without the requested meta_key, but it will do weird things when ordering.

add_action('pre_get_posts', 'my_stuff');
function my_stuff ($qry) {
    $qry->set(
        'meta_query',
        array(
            'relation' => 'OR', # Matches to this meta_query should be added to those matching the 'meta_key' query
            array(
                'key' => 'item_price', 
                'value' => 'bug #23268', 
                'compare' => 'NOT EXISTS'
            )
        )
    );

    $qry->set('orderby', 'meta_value date'); # Sorting works with meta_value as well as meta_value_num - I've tried both
    $qry->set('order', 'ASC DESC');
    $qry->set('meta_key', 'item_price');
}

I found this by fiddling around with all the different answers to this question and analyzing the generated SQL through trial and error. It seems that setting array('meta_query' => array('relation' => 'OR')) outputs an appropriate LEFT JOIN instead of INNER JOIN that is necessary to include posts missing the metadata. Specifying the NOT EXISTS prevents the WHERE clause from filtering out posts lacking the meta field. For this particular WP_Query, the generated SQL is (indentation/newlines added):

SELECT SQL_CALC_FOUND_ROWS
    wp_posts.ID
    FROM wp_posts
    INNER JOIN wp_term_relationships ON (wp_posts.ID = wp_term_relationships.object_id)
    INNER JOIN wp_postmeta ON wp_posts.ID = wp_postmeta.post_id
    LEFT JOIN wp_postmeta AS mt1 ON (wp_posts.ID = mt1.post_id AND mt1.meta_key = 'item_price')
    WHERE 1=1
    AND ( wp_term_relationships.term_taxonomy_id IN (2) )
    AND wp_posts.post_type = 'post'
    AND (wp_posts.post_status = 'publish'
        OR wp_posts.post_status = 'private')
    AND (wp_postmeta.meta_key = 'item_price'
        -- Oh look, here we give SQL permission to choose a random
        -- row from wp_postmeta when this particular post is missing
        -- 'item_price':
        OR  mt1.post_id IS NULL )
    GROUP BY wp_posts.ID
    ORDER BY wp_postmeta.meta_value,wp_posts.post_date DESC
    LIMIT 0, 10

The result is a listing of all the posts with meta_value of item_price and those missing item_price. All of the posts with item_price will be ordered correctly relative to each other, but posts missing item_price will use some random other meta value (say, _edit_last which seems to be 1 quite often in my database or some other internal wordpress metadata that is completely arbitrary) for its wp_postmeta.meta_value in the ORDER BY clause. So, while this method is close and may appear to work for certain data, it is broken. So, all I can say is, if your item_price values happen to not conflict with the random meta fields MySQL chooses for the posts missing item_price, this might work fine for you. If all you need is a guarantee that your posts with item_price are correctly ordered relative to each other without regard for the ordering of other posts, it may be OK. But I think this is just a shortcoming in wordpress. Please correct me, I hope I’m wrong and there’s a way to address this ;-).

It seems that for the INNER JOIN wp_postmeta, MySQL is choosing a random row from among multiple postmeta rows associated with the post when the meta_key is missing from the given post. From an SQL perspective, we need to figure out how to tell wordpress to output ORDER BY mt1.meta_value. This column is properly NULL when our requested meta_key is missing, unlike wp_postmeta.meta_value. If we could do that, SQL would sort these NULL (missing entries) before any other value, giving us a well-defined order: first come all the posts missing the particular postmeta field, second come the posts having the field. But that is the whole problem: 'orderby' => 'meta_value' can only refer to 'meta_key' => 'item_price' and the unaliased wp_postmeta is always an INNER JOIN instead of ever a LEFT JOIN, meaning wp_postmeta.meta_value and wp_postmeta.meta_key can never be NULL.

So I guess I have to say that this isn’t possible with wordpress’s built-in WP_Query as it is now documented (in wordpress-3.9.1). Bother. So if you actually need this to work correctly, you probably need to hook into wordpress elsewhere and modify the generated SQL directly.

  • Looks very promising! I will try this next time I have this issue. I'd like to give you the answer right now, but would prefer to confirm it works for me first. – powerbuoy Aug 5 '14 at 16:09
  • This kept everything from showing up for me. Nothing showed after implementing this. – Jake Oct 26 '16 at 11:32
  • @Jake Yea same here. Had this issue again today and tried this. Returns 0 results. – powerbuoy Jun 7 '17 at 9:00
  • What version of wordpress are you guys using? I think this post describes how to use an internal, undocumented API which is not supported by wordpress and so probably only works if you’re on wordpress-3.9.1 or not too many versions beyond that. – binki Jun 7 '17 at 14:40
8

Easy Peasy, just tested 2018, using in production currently.

$query->set( 'meta_query', array(
    'relation' => 'OR',
    array(
        'key' => 'custom_meta_key', 
        'compare' => 'EXISTS'
    ),
    array(
        'key' => 'custom_meta_key', 
        'compare' => 'NOT EXISTS'
    )
) );
$query->set( 'orderby', 'meta_value title' ); 

This checks for all items with and without the meta key, with no value specified. the meta query provides the key for the orderby reliably. It has been tested. I am not sure, however, how it will work when the meta query uses multiple keys.

Practical example

/**
 * Modifies query before retrieving posts. Sets the 
 * `meta_query` and `orderby` param when no `orderby` 
 * param is set, (default ordering).
 * 
 * @param   WP_Query  $query  The full `WP_Query` object.
 * @return  void
 */
function example_post_ordering( $query ) {

    // if not in wp-admin, 
    // and the query is the main query, 
    // and the query is not a singular query, 
    // and the query does not have an orderby param set...
    // Note: check for post types, etc. here as desired.
    if ( ! is_admin() 
    && $query->is_main_query() 
    && ! $query->is_singular() 
    && empty( $query->get( 'orderby' ) ) ) {

        // Setting just `meta_key` is not sufficient, as this 
        // will ignore posts that do not yet, or never will have 
        // a value for the specified key. This meta query will 
        // register the `meta_key` for ordering, but will not 
        // ignore those posts without a value for this key.
        $query->set( 'meta_query', array(
            'relation' => 'OR',
            array(
                'key' => 'custom_meta_key', 
                'compare' => 'EXISTS'
            ),
            array(
                'key' => 'custom_meta_key', 
                'compare' => 'NOT EXISTS'
            )
        ) );

        // Order by the meta value, then by the title if multiple 
        // posts share the same value for the provided meta key.
        // Use `meta_value_num` if the meta values are numeric.
        $query->set( 'orderby', 'meta_value title' );
    }

}

add_action( 'pre_get_posts', 'example_post_ordering', 10 );

This will order posts by custom_meta_key by default, and will not ignore posts without a value for that key.

  • Just from reading the code, all that seems to do is get posts which have custom_meta_key and get posts whick don't have custom_meta_key. Feel free to include an actual working example with sorting. – powerbuoy Apr 18 '18 at 17:33
  • 1
    You are right, that is all it is doing, but the line below is responsible for ordering by meta_value (of the meta key being queried). $query->set( 'orderby', 'meta_value title' ); (Order by meta value, then by title when multiple posts have the same value for the meta key). This should be done in the pre_get_posts hook, using the passed in $query variable. Keep in mind the question asked was how to order by meta value, while not ignoring posts that do not have a value for that meta key. – noahmason Apr 18 '18 at 18:18
  • @powerbuoy See updated practical example – noahmason Apr 18 '18 at 18:55
  • Alright I'll give it a go next time I'm facing this problem. – powerbuoy Apr 27 '18 at 18:13
  • 1
    Worked for me in a custom get_posts() call to shove posts with _featured meta to to the top, then order by date after that. Thanks! – natebeaty Nov 9 '18 at 18:03
2

I think I have a solution.

You can use two meta_keys, one that all posts have (like "_thumbnail_id"), and the meta_key you wish use as filter.

So your args:

$qry->set(
    'meta_query',
    array(
        'relation' => 'OR',
        array(
            'key' => 'item_price', 
            'value' => '', 
            'compare' => 'EXISTS'
        ),
        array(
            'key' => 'item_price', 
            'value' => '', 
            'compare' => 'EXISTS'
        )
    )
);

$qry->set('orderby', 'meta_value date'); # Sorting works with meta_value as well as meta_value_num - I've tried both
$qry->set('order', 'ASC DESC');
$qry->set('meta_key', 'item_price');
  • 1
    THe problem here is the empty string comparison, remove it and it works 'value' => '', also second compare should be NOT EXISTS and the last set instruction is not required – nodws Apr 24 '18 at 15:19
2

The problem that everyone here is having has to do with the order of the meta queries. In order to sort correctly, you will need to put the "NOT EXISTS" query before the "EXISTS" query.

The reason for this is because WordPress uses the meta_value of the last "LEFT JOIN" statement in the "ORDER BY" clause.

For example:

$pageQuery = new WP_Query([
    'meta_query' => [
        'relation' => 'OR',
        ['key' => 'item_price', 'compare' => 'NOT EXISTS'], // this comes first!
        ['key' => 'item_price', 'compare' => 'EXISTS'],
    ],
    'order' => 'DESC',
    'orderby' => 'meta_value_num',
    'post_status' => 'publish',
    'post_type' => 'page',
    'posts_per_page' => 10,
]);
1

If suitable, you can add a default meta value each time a post is saved or updated, if the meta value does no exists.

function addDefaultMetaValue($post_id) {
    add_post_meta($post_id, 'item_price', 0, true);
}
add_action('save_post', 'addDefaultMetaValue');

If you are using a custom post type, replace the add_action('save_post', 'addDefaultMetaValue'); by add_action('save_post_{post_type}', 'addDefaultMetaValue'); e.g. add_action('save_post_product', 'addDefaultMetaValue');

1

I had the problem by myself for numeric meta values and pointed out that it the order of the query is also important. For me the NOT EXISTS query has to be the first one.

Example:

$query->set( 'orderby', 'meta_value_num' );
$query->set( 'meta_query', [
    'relation' => 'OR',
    [ 'key' => 'your_meta_name', 'compare' => 'NOT EXISTS' ],
    [
        'key' => 'your_meta_name',
        'compare' => 'EXISTS',
    ],
] );

Also important in order to get the right direction for numeric values is the general ’orderby’ to be set to ’meta_value_num’. Otherwise you have strange results for numeric values, e. g.:

1, 2, 20, 21, 3, 4, 5 …

Instead of:

1, 2, 3, 4, 5 … 20, 21

1

I also encountered a similar problem and the following solution helped me:

$args = array(
'post_type' => 'kosh_products',
'posts_per_page' => -1,
'meta_query' => array(
    'relation' => 'OR',
    'category_sort_order' => array(
        'key' => '_sort_order',
        'compare' => 'EXISTS'
    ),
    'category_sort_order_not_exists' => array(
        'key' => '_sort_order',
        'compare' => 'NOT EXISTS'
    ), 
),
'orderby' => array( 
    'category_sort_order' => 'ASC',
    'date' => 'ASC'
));
$query = new WP_Query( $args );

I found a description on WordPress Codex with title "'orderby' with multiple 'meta_key's": https://codex.wordpress.org/Class_Reference/WP_Query#Order_.26_Orderby_Parameters enter image description here

0

There's a possible orderby value of meta_value for that.

$query = new WP_Query( array ( 
    'meta_key'   => 'your_keys_name',
    'orderby'    => 'meta_value',
    'order'      => 'DESC',
    'meta_query' => array( array(
         'key'     => 'your_meta_key',
         'value'   => '',
         'compare' => 'NOT EXISTS',
         // 'type'    => 'CHAR',
    ) )
) );

If you got numeric values, just use meta_value_num instead.

Disclaimer: This is not tested, but it should work. Point is that you need to specify your meta_key and key values. Else you can't compare against non-existing values, which should make it possible to query both sorts of posts. It's some kind of hack-ish, but as long as it works...

  • Thanks for your answer, please check my updated question, I'm not sure I understood you correctly. – powerbuoy Jun 10 '13 at 0:05
  • I still haven't made this work so if you have a solution I'd love to know what I'm doing wrong. Also, I set a bounty on SO if you want to claim it: stackoverflow.com/questions/17016770/… – powerbuoy Jun 20 '13 at 2:45
  • 1
    Two things. 'your_keys_name' and 'your_meta_key' should both be the same string instead of distinct, otherwise it looks like you’ve misunderstood the question. Secondly, I tested this on my local setup and it excludes any posts where the key exists (through the meta_query) and excludes any posts where the key is missing (through meta_key) resulting in no posts showing. However, this answer is a step toward something that words at least ;-). – binki Jul 31 '14 at 4:16
  • 1
    Oh, interestingly, this answer does work if you just add 'relation' => 'OR' to meta_query. Wacky stuff o_o. – binki Jul 31 '14 at 4:18
  • @binki Just file an edit to my question and change the bits you think should be changed. This is a community driven site :) – kaiser Jul 31 '14 at 9:58
0

I think what @kaiser was trying to do was to tell the query to return all posts that have that meta key by applying a sort of dummy where condition to not filter any of those posts. So if you know all the values your custom fields can take are x,y,z you could say "WHERE meta_key IN(x,y,z)" but the idea is you can avoid that problem all together by saying != (''):

$query = new WP_Query( array ( 
    'orderby'    => 'meta_value_num',
    'order'      => 'DESC',
    'meta_query' => array( array(
         'key'     => 'item_price',
         'value'   => '',
         'compare' => '!=',
    ) )
) );

Also not tested but feels like it's worth a try :-).

  • 1
    Can't really test this right now, but pretty sure that will only return posts where item_price is set and it's not ''. – powerbuoy Aug 23 '13 at 11:13
0

I ended up getting around this with a bit of a hack (IMHO), but it did the job for me in my case.

You can hook into the filters posts_join_paged and posts_orderby to update the join and order strings. This will allow you to order by whatever you want as long as you join it first rather than WP_Query assuming that the field has to exist for that particular post. You can then remove the meta_key, orderby, and `order from your WP_Query args.

Below is an example. At the top of each function I had to escape out for certain cases since it will add this to everything that uses WP_Query. You may need to modify that to fit your particular needs.

Documentation on these two filters is sadly lacking so... good luck! :)

add_filter('posts_join_paged', 'edit_join', 999, 2);
add_filter('posts_orderby', 'edit_orderby', 999, 2);

/**
 * Edit join
 *
 * @param string $join_paged_statement
 * @param WP_Query $wp_query
 * @return string
 */
function edit_join($join_paged_statement, $wp_query)
{
    global $wpdb;
    if (
        !isset($wp_query->query)
        || $wp_query->is_page
        || $wp_query->is_admin
        || (isset($wp_query->query['post_type']) && $wp_query->query['post_type'] != 'my_custom_post_type')
    ) {
        return $join_paged_statement;
    }

    $join_to_add = "
        LEFT JOIN {$wpdb->prefix}postmeta AS my_custom_meta_key
            ON ({$wpdb->prefix}posts.ID = my_custom_meta_key.post_id
                AND my_custom_meta_key.meta_key = 'my_custom_meta_key')
    ";

    // Only add if it's not already in there
    if (strpos($join_paged_statement, $join_to_add) === false) {
        $join_paged_statement = $join_paged_statement . $join_to_add;
    }

    return $join_paged_statement;
}

/** 
 * Edit orderby
 *
 * @param string $orderby_statement
 * @param WP_Query $wp_query
 * @return string
 */
function edit_orderby($orderby_statement, $wp_query)
{
    if (
        !isset($wp_query->query)
        || $wp_query->is_page
        || $wp_query->is_admin
        || (isset($wp_query->query['post_type']) && $wp_query->query['post_type'] != 'my_custom_post_type')
    ) {
        return $orderby_statement;
    }

    $orderby_statement = "my_custom_meta_key.meta_value DESC";

    return $orderby_statement;
}
  • The code works. But the meta_value is handled as a string. So 6 is rated higher as 50. Any modifications possible to treat them as numbers? – Drivingralle Jan 24 '17 at 17:59
  • @Drivingralle cast(my_custom_meta_key.meta_value as unsigned) DESC should do the trick... – tfrommen Feb 2 '17 at 21:34
  • 1
    Thanks @tfrommen. $orderby_statement = "cast(my_custom_meta_key.meta_value as unsigned) DESC"; works great. – Drivingralle Feb 6 '17 at 13:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.