The tag has no usage guidance.

learn more… | top users | synonyms (1)

0
votes
2answers
37 views

Robots.txt not updating

When I try to google search my web address, the follow comes up as the meta description "A description for this result is not available because of this site's robots.txt – learn more." The ...
0
votes
1answer
263 views

robots.txt Not Updating

Today I received an email from Webmaster Tools stating that "Googlebot cannot access CSS and JS files". So I set about updating my robots.txt file located in the root of my server. Original Contents: ...
0
votes
1answer
430 views

Disable Ajax for Spiders

We have a site on wpegnine that gets about 20k visits a day and heavily depends on ajax for our inbound marketing strategy (lead/visitor processing). These ajax calls are killing our server ...
0
votes
1answer
78 views

Hide pages from google and visitors

I have wordpress site where i have few custom type pages. I load the content of those pages with jquery load. But I dont want that those pages are directly visible. So if user goes to ...
0
votes
1answer
81 views

wp_redirect and search engine bots

I have some pages and categories you have to be looged in to have access to. I am using wp_redirect to send anyone who is not logged in to the front page if they try to open those pages. Is it enough ...
-2
votes
1answer
491 views

WP in Subdirectory — Multiple robots.txt and .htaccess files?

Assume you have a static HTML site with its own robots.txt and sitemap.xml at www.mysite.com. Now you add WordPress blog as an adjunct to the site at: www.mysite.com/blog Do you need to have a ...
0
votes
0answers
79 views

Robots.txt with WordPress and Joomla in same server

I have this configuration in my server root (hosted): root -> a Wordpress blog DirectoryJ -> a Joomla site DirectoryWP -> Another Wordpress blog I don't know how to handle the robots.txt. Joomla ...