I'm finding plugins that add the option for featured images for categories, but not how to code it yourself.
I had missed that you can easily add fields to taxonomy terms with Advanced Custom Fields: https://www.advancedcustomfields.com/resources/get-values-from-a-taxonomy-term/
There are already some solutions out there for adding custom fields to taxonomy. See: Add custom field to Category
You will see a 2016 update near the bottom that should help you.