I have a WordPress website, I submitted it to Google Webmaster and after submitting the sitemap, it's giving me error "URL blocked by robots.txt".

Please help, what should I do?

  • Can you please specify what URL is being blocked? the entire website? a CSS file? or? – Jack Johansson Feb 9 '17 at 11:18
  • Go to Crawl - Robots.txt Tester and paste the results here please ! – smartcat Feb 9 '17 at 14:00

Try to go Settings --> Reading, take a look if Search Engine Visibility is checked, and in this case unchecked. Now the robots.txt has disallow only wp-admin path

  • It's unchecked. – Joel Feb 9 '17 at 10:53
  • Read link – ClodClod91 Feb 9 '17 at 10:55

You can read this article.

Some from there:

  1. This error appears when Google is unable to crawl the specific URL due to a robots.txt restriction.
  2. Google is unable to crawl this URL for a variety of reasons. The following places may contain a statement that prevents crawling.
    • <meta name=”robots” content=”noindex” />
    • X-Robots-Tag: noindex
    • Disallow: [Path Or / ] (in robots.txt. file)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.