I've read many WordPress Security blog articles where the Security Experts are recommending some special steps to take care when somebody is concerned about their WordPress site's security. One of them is:
WordPress Security Tips:
Remove unnecessary plugins, that are not in use.
A plugin that has security holes, whether by code, structure or db connections, can be fatal for a site even if it's activated on a site. On the other hand, a well structured, well coded, and securely db-connected plugin may not have a security hole even when it's deactivated. So where's the issue exactly?
I have a site where there are some plugins I use occasionally. I actually don't want to delete them but when they are not needed I just deactivate them from the site. Do I need to delete them to secure my site and if so, why?