What you need to do is compare the values of the meta field `_wp_page_template`, which contains the page template selected for a single page with the available page templates. 

For this you need to construct an array of used templates, because you want the templates used by all the pages, similar as shown here:  

 - [Return all custom meta data for one custom post type](https://wordpress.stackexchange.com/a/45272/22534)

Use `array_unique` to get unique values.

Then you need to get the available page templates, as shown here:

 - [get page templates](https://wordpress.stackexchange.com/q/83180/22534)

Last but not least, you can use `array_diff` to compare the arrays of used and available templates, which subsequently gives you the unused templates.