Add your plugin with a capability argument.
So, if your plugin entry point is an admin page menu, you can use something like this:
add_menu_page(page_title, menu_title, capability, handle, [function], [icon_url])
You can set the "capability" to "upload_files". that function sees that capability argument as "The minimum capability required to display and use this menu page".
Maybe a good capability to use is publish_pages?
for more info:
http://codex.wordpress.org/Function_Reference/add_menu_page
http://codex.wordpress.org/Roles_and_Capabilities