Take the 2-minute tour ×
WordPress Development Stack Exchange is a question and answer site for WordPress developers and administrators. It's 100% free, no registration required.

I need some help with robots.txt.

Under Privacy Settings, I selected "I would like my blog to be visible to everyone, including search engines..."

As I understand it, WordPress automatically creates a robots.txt file. But, I need to exclude one page from the automatically generated robots.txt file. Let's say the page is called, "myPage". How do I keep myPage from being indexed?

If I make my own static robots.txt will it be overwritten by WordPress' dynamic one?

Also, does WordPress' auto-generated robots.txt exclude files like wp-admin, that shouldn't be indexed?

Thank you!

-Laxmidi

share|improve this question

1 Answer 1

up vote 1 down vote accepted

you can just make a file in your root named robots.txt if you don't already have one

User-agent: *
Disallow: /myPage

is the format for that one page there is no allow, only disallow. disllowing pages you follow the above format

WP may not have generated a robots.txt, I believe it only would have if you wanted your site not indexed

I'm not sure if WP has inbuilt mechanism for not indexing your admin stuff....but mine's never been indexed. If you are worried, to block entire directories you add

Disallow: /wp-admin/

it's one entry per line/disallow

share|improve this answer
    
Thank you for the help Rev. Voodoo. –  Laxmidi Apr 22 '11 at 16:39

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.