I have a site visibility in privacy settings set to allow search engines to see my site but for some reason when i go to the auto generated robots.txt file it shows

User-agent: * Disallow:

switching site visibility merely adds/removes a trailing slash ie "Disallow: /"

i have 2 related plugins "all in one seo tools" and "google xml sitemaps" but i disabled both of these to no avail

any ideas anyone .... i know how to override this with a manual sitemap but i'm trying to find out whether this is WordPress intended behavior and if so why

  • I think I found the answer to this. Apparently Disallow: with no trailing slash is the same as Allow All .... however google doesn't fully comply with this. There is a very good discussion here wordpress.org/support/topic/… Commented Aug 8, 2011 at 20:03

1 Answer 1


Disallow with no trailing slash is the same as "allow all" - see here - nice find!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.