I am not sure where exactly I navigate to add the analytics code once so it gets to all the WordPress pages.
Do I change the HTML manually or can I do this from WordPress somewhere?
WordPress Development Stack Exchange is a question and answer site for WordPress developers and administrators. It only takes a minute to sign up.Sign up to join this community
It is not recommended to edit the theme file directly. It is best practice to use some sort of plugin.
Monster Insight is one of the best available in the market. It is available in the WordPress repository.