This depends on your theme.
Most themes have a call the
wp_footer hook at the bottom of every page - which means it's at the bottom of every single post, every single page, and the home page. Rather than editing all of your theme files to include your tracking code, use a plugin that ties in to
wp_footer and adds the code there.
There's actually a plugin that will do this for you - which means you just install the plugin, no code editing is required.
Google Analytics for WordPress
I did test this exact plugin with a fresh install of Twenty Ten 1.2. It works just fine and displays the script on the home page. You must have another plugin interfering with the Google Analytics one or you've made changes to the theme that's preventing the plugin from working properly.