The tag has no usage guidance.

learn more… | top users | synonyms (1)

0
votes
2answers
60 views

Robots.txt not updating

When I try to google search my web address, the follow comes up as the meta description "A description for this result is not available because of this site's robots.txt – learn more." The ...
0
votes
1answer
497 views

Disable Ajax for Spiders

We have a site on wpegnine that gets about 20k visits a day and heavily depends on ajax for our inbound marketing strategy (lead/visitor processing). These ajax calls are killing our server ...
0
votes
1answer
80 views

Hide pages from google and visitors

I have wordpress site where i have few custom type pages. I load the content of those pages with jquery load. But I dont want that those pages are directly visible. So if user goes to ...
0
votes
1answer
84 views

wp_redirect and search engine bots

I have some pages and categories you have to be looged in to have access to. I am using wp_redirect to send anyone who is not logged in to the front page if they try to open those pages. Is it enough ...
-2
votes
1answer
524 views

WP in Subdirectory — Multiple robots.txt and .htaccess files?

Assume you have a static HTML site with its own robots.txt and sitemap.xml at www.mysite.com. Now you add WordPress blog as an adjunct to the site at: www.mysite.com/blog Do you need to have a ...
0
votes
0answers
81 views

Robots.txt with WordPress and Joomla in same server

I have this configuration in my server root (hosted): root -> a Wordpress blog DirectoryJ -> a Joomla site DirectoryWP -> Another Wordpress blog I don't know how to handle the robots.txt. Joomla ...
-1
votes
0answers
21 views

Googlebot is unable to Crawl my website

i am having this issue: "Google couldn't crawl your site because we were unable to access your site's robots.txt file." My site: TechGeekZone.com/robots.txt I have already tried all the suggestions ...