Worried I have a funky .htaccess for WP site in light of strange search engine behavior

Question

My friend started a new (non-commercial) blog 5+ months ago, but Google + Bing are acting like we're telling it to not be indexed.

I don't mean it ranks poorly. I mean it doesn't rank at all.

I checked Google Webmaster Tools. No malware/bad rep.

Google + Bing crawl it regularly, but index no pages.

I initially suspected robots.txt, but GWT is not complaining about it. And so now I turn to .htaccess.

Does this look strange to you? Feedburner having two entries? Deny from all and Allow from all in the same entry? I'm pretty ignorant about .htaccess and Apache, but just the inconsistency seems funny.

# temp redirect wordpress content feeds to feedburner
<IfModule mod_rewrite.c>
RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} !FeedBurner    [NC]
RewriteCond %{HTTP_USER_AGENT} !FeedValidator [NC]
RewriteRule ^feed/?([_0-9a-z-]+)?/?$ http://feeds.feedburner.com/anonymousblog  [R=302,NC,L]
</IfModule>

# temp redirect wordpress comment feeds to feedburner
<IfModule mod_rewrite.c>
RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} !FeedBurner    [NC]
RewriteCond %{HTTP_USER_AGENT} !FeedValidator [NC]
RewriteRule ^comments/feed/?([_0-9a-z-]+)?/?$    http://feeds.feedburner.com/anonymous_comments [R=302,NC,L]
</IfModule>

<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>

IndexIgnore .htaccess */.??* *~ *# */HEADER* */README* */_vti*

<Limit GET POST>
order deny,allow
deny from all
allow from all
</Limit>
<Limit PUT DELETE>
order deny,allow
deny from all
</Limit>

php_value memory_limit 32M

Answer 1

WordPress responds to requests to a robots.txt with dynamic content if such a file does not exist. That’s one way how the settings from wp-admin/options-privacy.php are used.

I recommend to create a static robots.txt, just to make sure no plugin is getting in your way.

Sample robots.txt

User-agent: *
Disallow: /cgi-bin
Disallow: /wp-admin
Disallow: /wp-includes
Disallow: /wp-content/plugins
Disallow: /wp-content/cache
Disallow: /wp-content/themes
Disallow: /trackback
Disallow: /comments
Disallow: */trackback
Disallow: */comments

User-agent: backlink-check.de
Disallow: /


# Prefetches everything. Mwaaah!
User-agent: Fasterfox 
Disallow: /

# adjust the path
Sitemap: http://example.com/sitemap.xml

Your .htaccess looks indeed … strange. You need RewriteEngine On just once. And a memory limit of 32MB is very low. You cannot even run translation with such a low value.

You should limit the request methods to HEAD, GET and POST.

Sample .htaccess

<IfModule mod_rewrite.c>
RewriteEngine on
RewriteBase /

# FeedBurner
RewriteCond %{HTTP_USER_AGENT} !FeedBurner    [NC]
RewriteCond %{HTTP_USER_AGENT} !FeedValidator [NC]
RewriteRule ^feed/?([_0-9a-z-]+)?/?$ http://feeds.feedburner.com/anonymousblog  [R=302,NC,L]
RewriteRule ^comments/feed/?([_0-9a-z-]+)?/?$    http://feeds.feedburner.com/anonymous_comments [R=302,NC,L]

# WordPress
# Existing file
RewriteCond %{REQUEST_FILENAME} !-f
# Existing directory
RewriteCond %{REQUEST_FILENAME} !-d
# Symbolic link
RewriteCond %{REQUEST_FILENAME} !-l
RewriteRule ^ index.php [L]
</IfModule>

IndexIgnore .htaccess */.??* *~ *# */HEADER* */README* */_vti*

<LimitExcept HEAD GET POST>
order deny,allow
deny from all
</LimitExcept>

php_value memory_limit 128M