"Template hierarchy" describes process by which WordPress determines which theme files to use when displaying a query. Index.php is the fallback for all queries, and, theoretically, the only theme file required other than style.css.

The WordPress Codex contains an extensive and well-written description of the Template Hierarchy. It includes an oft-cited diagram of the Template Hierarchy.

From the Codex:

WordPress Templates fit together like the pieces of a puzzle to generate the web pages on your WordPress site. Some templates (the header and footer template files for example) are used on all the web pages, while others are used only under specific conditions.

As an example, an archive page will use index.php to display itself. However, if the theme provides an archive.php file instead it will use that. If it's a Category Archive, the theme could use a file named category.php. To get even more specific, a specific category term's archive page can be styled using category-{slug}.php or category-{ID}.php.