Hi Matthew Paulson,
I see your using W3 Total Cache but your database and object cache is set to disk. Caching objects and your database to disk can actually have a negative performance effect especially if your getting that much traffic.
You can read more about the effects on caching database and objects to disk in an article I wrote on how to set up W3 Total Cache The plugin author agreed with the instructions in my settings.
To really see the benefit of database and object caching you need to be using a PHP opcode cache like APC. You can follow the copy and paste instructions in the plugin FAQ to compile and set up APC. If your on Ubuntu or Debian you can simply run the command: apt-get apc-php5 to install.
Like others have also mentioned you will get a huge performance boost and enable your site to scale much larger by setting up a reverse proxy with Nginx.
I give detailed instructions on how to configure and set it up in my WordPress Performance Stack. article.
You should also read some of the other questions and answers on here. A lot of good performance and scaling advice has been given.
Good luck on your quest. Managing your own server can be very stressful sometimes.
Just to show the performance you can gain by installing Nginx as a reverse proxy I'm posting an Apache Benchmark test I just ran on my server:
x-wing ~: ab -n 1000 -c 80 http://wp-performance.com/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking wp-performance.com (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests
Server Software: nginx/0.8.54
Server Hostname: wp-performance.com
Server Port: 80
Document Path: /
Document Length: 3132 bytes
Concurrency Level: 80
Time taken for tests: 0.066 seconds
Complete requests: 1000
Failed requests: 0
Write errors: 0
Total transferred: 3605000 bytes
HTML transferred: 3132000 bytes
Requests per second: 15164.15 [#/sec] (mean)
Time per request: 5.276 [ms] (mean)
Time per request: 0.066 [ms] (mean, across all concurrent requests)
Transfer rate: 53385.52 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 2 0.6 1 3
Processing: 1 4 0.8 4 5
Waiting: 1 3 0.8 3 5
Total: 3 5 0.6 5 7
Percentage of the requests served within a certain time (ms)
100% 7 (longest request)
Theoretically it's able to handle over 15,000 requests per second. (Same Network)