WordPress does not support a bilingual or multilingual blog out-of-the-box. There are however Plugins developed by the WordPress community which will allow you to create a multilingual blog easily.
Creating a mulitlingual blog is basically installing WordPress in more than one language and letting the Plugin switch between them. This includes installing .mo languages files which most Plugins will require you to do manually. See Installing WordPress in Your Language for details.
The free Polylang, qTranslate or xili-language plugins are installable on standalone WordPress sites. For multisite WordPress (one website per language), you can try Multisite Language Switcher or Multilingual Press or purchase WPML.