I have a client who is asking to format the RSS feed so the XML doesn't look so ugly. I was thinking of a plugin or something to detect the user-agent and modify the feed template based on a regular browser request as opposed to a reader that will need the raw XML. Does it sound like I'm on the right track? I was going to use something like feedburner, but this site has many authors and they add new ones all the time and don't want to have to setup something at feedburner for every new feed they get.
You won't need browser detection, RSS (as XML) can be styled without losing compatibility with machine readers. I believe the format for that is XSLT.
Unfortunately I have little hands on experience with XSLT and none in context of WordPress feeds, so can't give solid example.
Frankly, don't waste time re-inventing the wheel. Just go with an existing, automated tool like Feedburner. It takes seconds (literally) to set up a new feed. It also gives you stellar analytics so you know who's actually subscribing to the feed.
Feedburner will also automatically re-format the feed for human consumption, provide simple icons to subscribe to the feed in the user's feed reader of choice, and allows you to easily support email subscriptions without adding a second tool.
What I believe you're looking to do is create what's called in WordPress a "custom feed template."
Basically, just like you can have index.php?feed=rss2 or index.php?feed=atom you can create a new custom feed type index.php?feed=human_readable. It's not entirely straightforward, but basically you need to create this template in one file then in another plugin file hook it up to the WP rewrites.
The WordPress Google Sitemap plugin uses a custom feed template to create a sitemap and you might be able to use that as an example.