Typically - including in plug-ins with 100,000's of users - the only security measure is a nonce.
Now, to me, for this particular functionality, the nonce seems as though it ought to be sufficient: If a site has already been compromised to the extent that users with Admin privileges are entering malicious scripts on their own, then the situation seems rather hopeless...
Some developers seem to believe, however, that the CSS code on the way via a $_POST variable to a function processing it and adding it to the options table, and then to particular page headers, can be intercepted and modified. I don't understand how exactly that is supposed to happen, again presuming that the Admin is acting responsibly. Not saying it can't happen - just don't know. (Anyone care to explain?)
WordPress itself does not include a CSS sanitizer or validator, or CSSTidy-type function, but highly security-conscious developers will sometimes try to emulate Jetpack or WordPress Core practices, adding CSSTidy or CSS parser, and aiming to remove vulnerabilities. The question was discussed, and some methods or approaches were proposed or outlined here and here .
At the moment, I'm thinking of applying the following to the the $_POST processing function - the variable in question being "style":
$style = filter_input( INPUT_POST, 'style', FILTER_CALLBACK, array( 'options' => 'tidystyle' ) ) ;
And the called function would look something like this (the first four lines of the actual function coming from the first of the two prior StackExchange/Overflow questions linked above)
However, I wonder whether any of this - or the more developed CSSTidy/HTML Purify, etc., methods - is really necessary or advisable or achieves anything significant for this particular type of usage: Admin-level user adding custom CSS via textarea input.
(PS just noticed that the function called within the callback function -
_sanitize_text_fields - is apparently only since WP 4.7 - so be careful if you use it! )