For an general overview about hooks take a look at the codex page about the Plugin API:
This article is specifically about the API of "Hooks", also known as "Filters" and "Actions"
Besides that there is valuable information specifically about customizing the login and registration form available at the codex:
Customizing the Login Form
You can customize your WordPress login page with action hooks and
filter hooks, including:
- Actions in the of the document: login_enqueue_scripts, login_head.
- Filters in the : login_headerurl, login_headertitle, login_message, login_errors.
- Actions at the bottom of and below the form: login_form, login_footer.
Customizing The Registration Form
Theme and plugin developers can customize WordPress's built-in user
registration page through the use of hooks.
Customizing the registration form involves utilizing the following
Allows rendering of new HTML form elements.
Perform validation on form registration fields.
Save custom form data.
The information given above should get you started learning more about customizing those forms - besides that there are some good examples available too. Especially you get an first overview about the hooks you can use.
But those aren't all there are, so additionally take a look at the other available hooks for Adminstrative Actions you can use for further customizations of those forms. Not all of them are documented well (or at all), so it might be necessary to take a look at the source, for example wp-login.php and general-template.php - those are the sources for the 3.6 branch.