with interpreted languages like php every line added to the code means more memory consumption, so it is trivial that with every feature added WP requires more memory.
You don't specify when you got that message, and if you are running a non english WP. I think that many strings were added as part of the builtin help system, and if you run a translated WP you load the translation of them as well.
If you upgraded only the WP then it is unlikely that the fault is with a theme or plugin. It is hard to measure how much memory each plugin and theme uses as most of it is probably allocated dynamically and changes between different front end/admin pages on the site. For example less memory will be consumed on single post page then on category page as there is simply less data to process.
If memory is an issue, and you can make changes to the server then you should install apc or other php accelerator. Since they precompile the php code into byte code and it takes less memory (and executes faster).
BUT.... I'm totally not sure you are going to run into a problem if you set WP_MEMORY_LIMIT on all of your sites. WP_MEMORY_LIMIT just say to WP (and WP communicates it to the php interpreter) that the maximal memory consumption can be 128k, but WP doesn't allocate this amount of memory on start up, and unless something is totally wrong with your plugins and theme, most of the time you are well below it.
So the important thing to know is what specific operation causes the high usages of memory. If it is only in admin then don't worry about it. For example, a very simplistic and inaccurate calculation - if you have 512k allocated to your server you will be able to handle 4 max memory page requests at the same time, but what is the chance of 4 admin pages being accessed at exactly the same time (the exact same second)? very close to 0 for most WP sites.
And there is even easier thing you can do - in you php.ini file set memory_limit to 128k. this way you will not need to worry about adding the WP_MEMORY_LIMIT define for every new site you install.