I'm looking for a way to allow Wordpress (via a plugin?) to assign a specific post format to categories. (I can assign categories via a third party like IFTTT)
Then all my posts in a certain category will be formatted rather than just text posts.
Please see this answer: Plugin to set all Posts in a certain Category to a certain Post Format which provides a way to do this by hard-coding it into your theme.
This should allow you to assign post formats to a certain category. You could also go the route of adding a categories button to a specific post format.