The tag has no wiki summary.

learn more… | top users | synonyms (1)

6
votes
6answers
2k views

What is a good robots.txt?

What is the "best" setup for robots.txt? I'm using the following permalink structure /%category%/%postname%/. My robots.txt currently looks like this (copied from somewhere a long time ago): ...
0
votes
2answers
2k views

Add robots.txt to root

I have a new Wordpress 3.5.1 install (hosted on Dreamhost FWIW) that I do not want to be indexed by search engines. I would like to serve a simple robots.txt with Disallow: / for all user agents. I ...
0
votes
1answer
72 views

Hide pages from google and visitors

I have wordpress site where i have few custom type pages. I load the content of those pages with jquery load. But I dont want that those pages are directly visible. So if user goes to ...