It's very possible, but quite hard to accomplish reliably for all cases (I was working on a plugin for client to do this and more, but sadly the project was never completed).
But if you need this for specific blog that you have control over - then task can be simplified with some constrains - mainly limiting processing to attachments only.
If so:
- filter
the_content_rss
, extract all images with regexp;
- determine attachments IDs for images, see this question and my answer there for some helpful code;
- retrieve attachment
img
tag in smaller size and replace currently used version with it.