A lot of this is very subjective and hard to answer because of different server environments, themes, size of database etc.
In your experience, when is the time one needs to start thinking about scaling up?
If you are concerned with your users experience you should already be practicing sound front end performance techniques. When is it time to scale up? When your users experience starts degrading and slow page loads are causing your bounce rate to increase.
What is the expected performance of a common WordPress installation on a standard Apache web server, without any performance tweaks or plugins?
A common WordPress installation running the default theme for a small to medium blog on one of the recommended shared hosts should be able to handle hundreds of users a day without issue.
In terms of page loads (let's assume no caching): Is it in range of 100's/1000's/+ of page loads per second?
For this question I ran a Apache Benchmark on a WordPress installation running the default 2010 theme with no caching.
I made 500 requests at 10 requests per second and was able to average 3.6 requests per second but as the requests started piling up the longest request took almost 2 minutes.
In terms of traffic: At which number of users per day can one expect to start hitting performance bottlenecks (ignoring traffic, assuming great connection)?
It is impossible to answer this question without running load tests or benchmarks on the server. For the benchmark above you would start having bottlenecks anytime there were more than 10 users using the site at once.