I work on a Load balanced instance of WordPress where all uploaded content is stored on a separate server and each WordPress server's upload or blogs.dir folder is linked to that content server (a shared disk).
Out of the box I do not believe there is way to force Wordpress to use a single server. You could try some host file magic and force the admin to use HTTPS and then limit all SSL connections to that server, but I would not recommend this solution.
If you need more help, you will need to provide more information about your set up as scribu mentioned.