Does a website slow down or under perform in any way with a large number of widgets installed?
Put simply: not inherently. If you have a widget that does NOTHING at all...like, literally you add it and there's no code for the widget outside of registering it, you'll see SUCH a small performance hit from adding it that you would need to be looking at
microtime() numbers to notice anything.
That said, as @cmegown hit on a little, it depends what the widgets do. If you have a widget that calculates pi or solves differential equations or something heinous like that (I can't really think of a good example), then you're gonna start looking at more and more of a performance hit.
Real world, where you're going to see issues is widgets that make HTTP requests, be those feeds, stylesheets, or calls to cURL. The best advice I can give you is to know what your widgets are doing, and make sure they're doing it right. Typically if you do that, everything else falls into place.
I highly doubt you'll run into any significant performance problems with a lot of simple widgets. However, you start putting widgets that grab twitter feeds and facebook feeds all over the place - you'll start seeing some issues. Since that's obviously not a very smart thing to do, I wouldn't worry about it.
The only other thing worth mentioning is that I would avoid downloading plugins to acquire more widgets, especially from a source other than wordpress.org. They are frequently riddled with malicious code that could easily effect your sites performance.
There's two ways to check that I know of. First is simply test page load time. Second is a little more involved but gives you actual statistics. Set SAVEQUERIES = true in wp-config.php, get the Debug Bar plugin (adds the debug information into an ajax dropdown in the top bar), and take a look at how long the server is taking to pull everything it needs. Widgets only seriously slow things down if they're pulling info from external sites or they're badly coded though, so I wouldn't worry too much about it.
There's a plugin called P3 (Plugin Performance Profiler) that helps you checking the performance hit in comparison to the rest of your install.