The WordPress template functions don't support passing variables (to my knowledge) by default, so you need to write your own function. For example like this,
// functions.php
function my_template_include_with_variables( string $path = '', $data = null ) {
$file = get_template_directory() . '/' . $path . '.php';
if ( $path && file_exists( $file ) ) {
// $data variable is now available in the other file
include $file;
}
}
// in a template file
$critical_css = 'style';
// include header.php and have $critical_css available in the template file
my_template_include_with_variables('header', $critical_css);
// in header.php
var_dump($data); // should contain whatever $critical_css had
UPDATE 18.8.2020
Since WordPress 5.5 you are able to pass additional arguments with the native template functions (e.g. get_header(), get_template_part(), etc.) to the template files in an array.
From dev docs,
get_template_part( string $slug, string $name = null, array $args =
array() )