I have a WordPress website, I submitted it to Google Webmaster and after submitting the sitemap, it's giving me error "URL blocked by robots.txt".
Please help, what should I do?
WordPress Development Stack Exchange is a question and answer site for WordPress developers and administrators. It only takes a minute to sign up.Sign up to join this community
Try to go Settings --> Reading, take a look if Search Engine Visibility is checked, and in this case unchecked. Now the robots.txt has disallow only wp-admin path
You can read this article.
Some from there:
- This error appears when Google is unable to crawl the specific URL due to a robots.txt restriction.
- Google is unable to crawl this URL for a variety of reasons. The following places may contain a statement that prevents crawling.
- <meta name=”robots” content=”noindex” />
- X-Robots-Tag: noindex
- Disallow: [Path Or / ] (in robots.txt. file)