get available brand attribute in woo commerce shop page, category page

Question

I used below code to display the brands in shop page, product category page's sidebar.

function custom_register_brand_taxonomy() {
    $args = array(
        'hierarchical' => true,
        'label' => 'Brands',
        'query_var' => true,
        'rewrite' => array( 'slug' => 'brand' ),
    );

    register_taxonomy( 'pa_brand', 'product', $args );
}
add_action( 'init', 'custom_register_brand_taxonomy' );


function custom_brand_filter_widget() {
    register_widget( 'Custom_Brand_Filter_Widget' );
}
add_action( 'widgets_init', 'custom_brand_filter_widget' );

class Custom_Brand_Filter_Widget extends WP_Widget {

    function __construct() {
        parent::__construct(
            'custom_brand_filter_widget',
            'Brand Filter',
            array( 'description' => 'Display a brand filter for WooCommerce products' )
        );
    }

    public function widget( $args, $instance ) {
        echo $args['before_widget'];

        // Display your brand filter HTML here as checkboxes
        // You can use get_terms to retrieve the brand terms and generate checkboxes
        // Example:
        $taxonomy = 'pa_brand';
        $terms = get_terms( array(
            'taxonomy' => $taxonomy,
            'hide_empty' => true,
        ) );

        if ( ! empty( $terms ) ) {
            echo '<div class="bapf_sfilter">';
            echo '<ul>';
            foreach ( $terms as $term ) {
                echo '<li>';
                echo '<input type="checkbox" name="' . esc_attr( $taxonomy ) . '[]" value="' . esc_attr( $term->slug ) . '"> ';
                echo esc_html( $term->name );
                echo '</li>';
            }
            echo '</ul>';
            echo '</div>';
        }

        echo $args['after_widget'];
    }

    public function form( $instance ) {
        // Widget form fields
    }

    public function update( $new_instance, $old_instance ) {
        // Update widget settings
    }
}

currently, all the brands are displayed on the category page. some category products might not have a specific brand. so if there is no any product available for that brand, I need to hide in the sidebar. for example, if it is bakery category, only the bakery category brand should show. drinks related brand should not get displayed. How can i achive that.