If the site is deleted by the site administrator, then the site is only marked as deleted (it receives the "deleted" attribute). Also the users are removed from the site (this might be changed in future WP versions). All database tables and files remain.
If the Super Admin deactivates the site, it is marked as deleted (it receives the "deleted" attribute) but the users are not removed from it. All tables and files remain.
If the Super Admin deletes the site from the Network Admin Sites Screen with the red Delete link, all default WordPress database tables are deleted, the ones created by plugins remain, unless the plugin is made to delete them on site deletion. The users are removed from the site. The media files of the site are also deleted. There is a situation though in which the files and tables may remain, if WordPress cannot reliably determine the uploads folder.
If you want to manually delete a site you can delete all database tables that start with the main site prefix followed by the site ID and then underscore. So for site ID 33 and main prefix "wp_" you get "wp_33_" (without quotes). Also go inside either the wp-content/blogs.dir or the wp-content/sites folder and delete the folder with the site ID number. And also delete from the wp_blogs database table the row with the ID of the site in column blog_id (not site_id). There is also some user data in the wp_usermeta table. All rows where the meta_key column value starts with the same prefix we talked about ("wp_33_" for example). If you want to be able to restore this site, I guess you can save a copy of all these things that you deleted and put them in the same place to restore it.
If you want a plugin that will properly delete all database tables of the site, I have added this feature to my premium plugin called Beyond Multisite. It has a lot of features, but one of the checkboxes in the settings of the Improvements module, is called "Delete any leftover database tables when a site is permanently deleted". If it is checked, when a site is deleted with the red Delete link in the Network Admin Sites Screen, all tables that start with the prefix of the site (for example "wp_33_") will be deleted (not only the default ones). Also using the Cleanup module of the plugin you can delete all leftover database tables of previously deleted sites.
In regards to plugins and themes they will not be deleted by any of the methods of site deletion. They are global and do not exist on a per site basis. You can delete the ones that you are not using from the Network Admin Plugins Screen or the Network Admin Themes Screen. Using my plugin, mentioned above, you can check if a plugin or a theme is activated in any sites (and which), which is always useful when you want to delete them. Just click the "Activated in?" action link under a plugin or a theme.