get_query_var will get the category from the main query, and it is only going to be set for some pages, such as category archives. It is not set for "single" pages or tag archives or a many other kids of "pages".
The value that
get_query_var returns is something like a 'search' variable. It will be set when the query is instructed to look for posts in a particular category. It doesn't reflect the value of current post in the Loop, though of course if
get_query_var('cat') is set then all posts should be assigned to that category.
What you want to get the category for the current post in the Loop is
get_the_category, but it should come after
get_the_tags will do the same for tags and the more generic
get_the_terms can be used for any taxonomy.