Is there a simple (and without plugin) way to clean up the WordPress output of HTML? With a beautiful indentation of the code? I don't understand why, but my HTML output isn't perfect and it makes me uncomfortable.
If I'm correct, you want to display HTML in your post output with indentation. There is no inbuilt way of doing it. You have to do some manual work, here's a link to a tutorial. This will help you understand how to write code in WordPress blog posts without using any plugins.
Check out the WordPress Codex too.