The tag has no wiki summary.

learn more… | top users | synonyms (1)

0
votes
1answer
74 views

Disable Ajax for Spiders

We have a site on wpegnine that gets about 20k visits a day and heavily depends on ajax for our inbound marketing strategy (lead/visitor processing). These ajax calls are killing our server ...
0
votes
1answer
60 views

Hide pages from google and visitors

I have wordpress site where i have few custom type pages. I load the content of those pages with jquery load. But I dont want that those pages are directly visible. So if user goes to ...
0
votes
1answer
254 views

How to solve Robots.txt problem with WordPress fresh install?

I installed a fresh copy of WordPress few weeks ago and I have been having some strange problems. 1- I get this message : "Comments for Yad Faeq yadfaeq.com/‎ A description for this result is not ...
0
votes
1answer
65 views

wp_redirect and search engine bots

I have some pages and categories you have to be looged in to have access to. I am using wp_redirect to send anyone who is not logged in to the front page if they try to open those pages. Is it enough ...
-2
votes
1answer
300 views

WP in Subdirectory — Multiple robots.txt and .htaccess files?

Assume you have a static HTML site with its own robots.txt and sitemap.xml at www.mysite.com. Now you add WordPress blog as an adjunct to the site at: www.mysite.com/blog Do you need to have a ...
0
votes
0answers
26 views

Unwanted “crawl delay: 10” line added to my robots.txt

I've noticed that when I make a request to my robots.txt page, I get it with a crawl delay: 10 line added to it. I double-checked the file, and it doesn't have that line, so why does it appear? Is it ...