WP_Query – order with usort by custom meta

Question

I have a custom post type location which has different categories (e.g. museums, theatres, shopping) and I want to output each of these categories and order within these categories by distance which is a custom meta field. Right now, I have following code, but cannot make the ordering work by distance.

<?php
        $custom_terms = get_terms('location_category');

    foreach($custom_terms as $custom_term) {
        wp_reset_query();
        $args = array('post_type' => 'location',
            'tax_query' => array(
                array(
                    'taxonomy' => 'location_category',
                    'field' => 'slug',
                    'terms' => $custom_term->slug,
                ),
            ),
         );

         $loop = new WP_Query($args);
         if($loop->have_posts()) {;?>

At the moment, I'm using usort to order alphabetically (see below), but this needs to be changed to ordering by distance.

    <?php usort($loop->posts, function($a, $b) {
   return strcasecmp( 
                $a->post_title, 
                $b->post_title 
            );
});?>

Can anyone please help me with this?