What is the "best" setup for robots.txt? I'm using the following permalink structure /%category%/%postname%/. My robots.txt currently looks like this (copied from somewhere a long time ago): ...
The homepage on a site we've developed has four pieces of content: the parent page and three further "pages" of text which we bring into the parent page using PHP. Call them Foo, Bar1, Bar2, Bar3 if ...
I have a new Wordpress 3.5.1 install (hosted on Dreamhost FWIW) that I do not want to be indexed by search engines. I would like to serve a simple robots.txt with Disallow: / for all user agents. I ...
I have wordpress site where i have few custom type pages. I load the content of those pages with jquery load. But I dont want that those pages are directly visible. So if user goes to ...