I have a site visibility in privacy settings set to allow search engines to see my site but for some reason when i go to the auto generated robots.txt file it shows
User-agent: * Disallow:
switching site visibility merely adds/removes a trailing slash ie "Disallow: /"
i have 2 related plugins "all in one seo tools" and "google xml sitemaps" but i disabled both of these to no avail
any ideas anyone .... i know how to override this with a manual sitemap but i'm trying to find out whether this is WordPress intended behavior and if so why