For reference the file that handles the tracking and usage statistics for Yoast SEO is located at,
I have linked to the GitHub repository file in question for further inspection upon which you can see somewhat, relatively, harmless collection of data.
However what you determine as "harmless" is case dependent because the opposite could be true if you are storing any sensitive data within the descriptive headers of plugin or themes that may be intended for internal use only.
What information does Yoast SEO gather about your installed themes,
'name' => $theme_data->display( 'Name', false, false ),
'theme_uri' => $theme_data->display( 'ThemeURI', false, false ),
'version' => $theme_data->display( 'Version', false, false ),
'author' => $theme_data->display( 'Author', false, false ),
'author_uri'=> $theme_data->display( 'AuthorURI', false, false ),
...and your installed plugins,
'version' => $plugin_info['Version'],
'name' => $plugin_info['Name'],
'plugin_uri' => $plugin_info['PluginURI'],
'author' => $plugin_info['AuthorName'],
'author_uri' => $plugin_info['AuthorURI'],
...and your site,
'site' => array(
'hash' => $options['hash'],
'url' => site_url(),
'name' => get_bloginfo( 'name' ),
'version' => get_bloginfo( 'version' ),
'multisite' => is_multisite(),
'users' => count( get_users() ),
'lang' => get_locale(),
'pts' => $pts,
'comments' => array(
'total' => $comments_count->total_comments,
'approved' => $comments_count->approved,
'spam' => $comments_count->spam,
'pings' => $wpdb->get_var( "SELECT COUNT(comment_ID) FROM $wpdb->comments WHERE comment_type = 'pingback'" ),
Determine if any of that information bothers you being in the hands of another before deciding whether or not to share tracking statistics.
The tracking script runs once per week as per this transient.
set_transient( 'yoast_tracking_cache', true, 7 * 60 * 60 * 24 );
Will that slow down your site? No. If it does, your problems are bigger than the tracking script then ;)
The benefits of providing tracking statistics is that the information sent upstream back to the developer can be used to improve the plugin and its compatibility with other plugins and themes overall making for a better all round plugin.
Its common practice.
The potential downside to this is that the developer knows what themes and plugins you have installed, also, a collective of other site details as shown above.
As much as this information can be used for developmental purposes, it could also be used for marketing purposes, not so much directly, but indirectly. Is that a bad thing? Not necessarily if the intention is good for which you'd have to place some faith in Yoast not wanting to stake his reputation over something as trivial.