Ajax infinite scroll random order shows duplicate posts on custom post type

Question

I have a custom post type which I want to show in a random order and with loading more posts with ajax infinite scroll. The problem is that I get duplicated posts on page two and onwards. I do understand that this is how WordPress handles random but I was wondering if I could use a session variable to store the posts from page 1, 2 ...

For my ajax and infinite scroll (and some unrelated filter) I use the plugin searchandfilter pro.

So far I managed to come up with this code in my functions.php but it doesn't seem to work as I still see duplicated on page 2 and so on:

add_action('init', 'start_session', 1);

function start_session() {
    if (!session_id()) {
        session_start();
    }
}


add_filter('sf_apply_filter_sort_post__in', 'sort_result_ids', 20, 3);

function sort_result_ids($result_ids, $query_args, $sfid) {
        // Check if the excluded_posts array exists in the session
        $excluded_posts = $_SESSION['excluded_posts'] ?? array();
        // Remove any excluded posts from the result IDs
        $result_ids = array_diff($result_ids, $excluded_posts);
        // Shuffle the result IDs
        shuffle($result_ids);
        // Store the new set of excluded_posts in the session
        $excluded_posts = array_diff($excluded_posts, $result_ids);
        $_SESSION['excluded_posts'] = $excluded_posts;
    
    return $result_ids;
}

Any ideas?