Template tag is defined the following here:
Template tags are used within themes to retrieve content from your database.
The loop retrieves (posts) content from database in templates, so it seems she would fall in that category. Hence I ask:
Is "the loop", the loopish function that shows posts while we have posts, based on if we have posts, should actually be considered a a template tag?
The Template tag is just a name chosen for a variety of function. It doesn't really matter what you call them, and they are functions.
However, the while( ) loop itself is a simple PHP feature. It uses the have_posts method of the WP_Query() class to detect whether there is any posts to display or not. The main query itself is already set when you load a page, which can be accessed via global $wp_query;.
So, putting all together, this can be an opinion based question. The WP_Query() class doesn't exists in the File section of Template Tags page, so I would call it a "No".
