As usual with hooks this is issue of timing.

• your init function is hooked to admin load process, which works fine for most things;
• however in this specific case function is added to filter in constructor of WP_Comments_List_Table class, and object is created in edit-comments.php after admin loader had been processed.

In my plugin for similar stuff I am removing it at manage_edit-comments_columns hook.

As usual with hooks this is issue of timing.

• your init function is hooked to admin load process, which works fine for most things;
• however in this specific case function is added to filter in constructor of WP_Comments_List_Table class, and object is created in edit-comments.php after admin loader had been processed.

In my plugin for similar stuff I am removing it at manage_edit-comments_columns hook.

Simplified example from my class:

static function on_load() {

    add_action( 'admin_init', array( __CLASS__, 'admin_init' ) );
}

static function admin_init() {

        add_filter( 'manage_edit-comments_columns', array( __CLASS__, 'manage_columns' ) );
}

static function manage_columns( $columns ) {

    remove_filter( 'comment_author', 'floated_admin_avatar' );

    // stuff

    return $columns;
}