I have a site with a pretty large media library. The images are mostly in JPG format. When I ran the site through Google's PageSpeed, it heavily penalized the score because the images were not optimized. I've tried a few plugins like Smush, but the PageSpeed still says the images need to be optimized. Which plugin or solution can optimize those images enough improve the score?
After trying a few image optimization plugins resmush.it did the trick. If you reduce the JPG quality to 70% Google should stop complaining and images still look good. The plugin is even free. Of course your images also have to be scaled properly.
The best way is to blatantly ignore it, unless your score is very low.
The reality of developing dedicated sites in which every piece of code is inspected, approved and controlled by you, is very different from the reality of 99.9% of wordpress sites. The reality of wordpress is that you have no control of what the core does and very little control on what the theme and plugins do.
This is a compromise that you do when you decide to use wordpress, as the maintenance of the site is not only in your hands but also depends on updates of the core themes and plugins, therefor it will be very hard to optimize any part of the code in them without risking breaking the possibility to upgrade them in the future.
In theory, you can adjust the theme, and configure wordpress core to create images in exactly the sizes that the theme/plugins/content use, but it is just not worth the effort IMHO.
There are few things to pay attention, but you need to understand the reason why Google complains.
It may be your theme is not smart. What this means it may scale big images to fit instead showing the smaller version of the images.
Just let me know if you need more info.
There is another level.
A web server approach where in Linux, you have great options.
EXIF tool to remove bad meta information. You just need to set the cron job once a day.
Since you need to optimize the existing images.
pngimages you have
Typically all these can be done via SmushIt plugin, but since you failed on that try the first part of my answer.