Questions about Robots.txt

Question

I need some help with robots.txt.

Under Privacy Settings, I selected "I would like my blog to be visible to everyone, including search engines..."

As I understand it, WordPress automatically creates a robots.txt file. But, I need to exclude one page from the automatically generated robots.txt file. Let's say the page is called, "myPage". How do I keep myPage from being indexed?

If I make my own static robots.txt will it be overwritten by WordPress' dynamic one?

Also, does WordPress' auto-generated robots.txt exclude files like wp-admin, that shouldn't be indexed?

Thank you!

-Laxmidi

Answer 1

you can just make a file in your root named robots.txt if you don't already have one

User-agent: *
Disallow: /myPage

is the format for that one page there is no allow, only disallow. disllowing pages you follow the above format

WP may not have generated a robots.txt, I believe it only would have if you wanted your site not indexed

I'm not sure if WP has inbuilt mechanism for not indexing your admin stuff....but mine's never been indexed. If you are worried, to block entire directories you add

Disallow: /wp-admin/

it's one entry per line/disallow