I just found it out!

By using get_query_var ( 'cat' ) you get the value of the category in the query.
So you can use cat_is_ancestor_of ( parent-cat-id, get_query_var ( 'cat' ) ) which will return true if you are looking at a child of a specific parent category.
And from there you can modify your archive page to load whatever templates they need.