Everything is in the title. I have no robots.txt on my website. I've enabled search engine indexing in the global settings. But still, Google cannot crawl my website due to an non-existing robots.txt.
Have you already experienced such an issue ?
1 Answer
Despite the fact your site lacks a robots.txt file, it might be worth making one to allow for Googlebot and other bots to crawl your website. Please use this example:
User-agent: *
Allow: /
Disallow: /wp-content
Note: Please remove or amend the /wp-content disallow on the robots.txt, if for whatever reason, you'd like your images indexed in search results.