Assuming that you are using WP 3.2+ and an up-to-date theme, the editor-style.css file should contain the styles for the editor. If you don't have one, just create it in the same place as style.css The style appear in the Styles dropdown not the paragraph type dropdown that you referenced in the question. This does indeed mean that you need to expand the button list using TineMCE advanced or one of the similar plugins (or you can do the configuration yourself in TinyMCE but it seems hardly worth it really).
You can also include all of the styles from your actual theme with:
This makes ALL of the styles available which may be rather overkill & may sometimes make the editor unusable because of style clashes.