I created a page on a Wordpress site that was for internal use only and triggers some backend code. Within a few days I started seeing hits on that page from "bingbot".
I'm not using any kind of sitemap plugin. How are crawlers finding this page?
I know the robots.txt file can block them but I want to make sure they don't show up for crawlers that don't respect this. I still want to have the page publicly accessible if someone types in the URL.
What needs to be done in Wordpress to make sure a page can't be discovered except by typing in the URL?