Take a look at the
add_settings_error( $setting, $code, $message, $type );
The first argument is your settings name/key -- or if your setting is on another page (eg
general) it should be the page key. The second is whatever you'd like to add to the ID attribute, then error/updated message, and finally type. It doesn't work because you're using it incorrectly.
So you probably want...
'your_setting_key', // whatever you registered in `register_setting
'a_code_here', // doesn't really mater
__('This is the message itself', 'wpse'),
'error', // error or notice works to make things pretty
You also need to tell WordPress to display your settings errors. If it's on a custom page, you'll need to include
settings_errors in the callback.