(This is an opinionated and subjective answer as your question involves many factors and doesn't include detailed information on the scenario)
You may be getting to the point where you have enough volume that you need more professional server optimisation, or you need to get stuck into more advanced server administration.
If you're not CPU or RAM bound then you're disk bound - probably MySQL disk access, but could be other things. A process doesn't have high usage if it's waiting for the disk. You can keep scaling your CPU and RAM up but that won't make the disk go faster so it won't make any difference, so CPU/RAM percentage utilisation will go down without the server speeding up. Check out answers like this for diagnosing beyond CPU and RAM, but note that it's probably MySQL and you should look at that first.
Quick wins in this situation:
- Because 90%* of Wordpress speed problems are related to big or poorly written database queries you need to look at the database first
- Use Query Monitor to try to understand if you have specific queries that are generating your load. Delays shown here will obviously be more prevalent and noticeable when your site is under load.
- In the big picture, it's a reasonably easy job and will give you performance and manageability gains to move MySQL to a separate server. This will let you scale MySQL independently from apache/php. This is probably what you should do first
- Try to speed up your disk access with easy wins, e.g. is your disk SSD?
- If it's not MySQL then maybe you have a plugin or other code that's doing some huge disk oeprations and you need to optimise it somehow.
Things which take a bit more time but will get you better at server admin:
- Get more familiar with htop and tools like iotop to see what's going on on your server.
- Assuming problem is in MySQL, figure out how to turn on the slow query log so you can get feedback and see again in the future where the bottlenecks are when things get slow.
- Yes there are a ton of settings to optimise in MySQL. Once you know a bit more about what's going on (e.g. rule out really bad queries you might be able to fix) then post in a place with more specific MySQL expertise for help
- When you know a bit more post about what's happening in e.g. serverfault where you'll get more specific optimisation advice