The Codex is a good place to start:
wp_head() function call goes in the head section of your theme template, and simply does all of the actions hooked to it. If you look in
/wp-includes/default-filters.php, you'll see all of the default actions that are called when this function is invoked.
You can also hook your own functions to do things or output data into the head of your theme. The same goes for
wp_footer(), which is for outputting data or doing background actions that run just before the closing body tag.
You hook an action to a function by using
add_action. Here's a simple example that would go in your theme's
functions.php file, or a plugin. Say you need to ouput a conditional comment for the IE6 browser:
<!--[if IE 6]>
Special instructions for IE 6 here
wp_head() is called in the theme,
your_function() is run, and the output will appear wherever that
wp_head() call is.
You can also remove action hooks, using
remove_action. For example, if you want to remove the feed links WordPress places in the head, you can remove that action:
remove_action('wp_head', 'feed_links', 2);
WordPress has many actions (see action reference) which are run at every stage of the execution of both a front-end as well as an admin request. This is basically the foundation upon which all plugins and many theme features function.
Have a read through the Plugin API for more info.