I am using WordPress SEO plugin written by Yoast. This plugin, by default, shows "noindex,follow" on every page. I am sure I know the file that displays "noindex, follow" but after much "tweaking" I can't seem to remove the code.

Here's the file: https://plugins.svn.wordpress.org/wordpress-seo/tags/

I'd really appreciate if you can let me know which line needs to be commented out, or what needs to be edited. :)

  • Have you visited the plugin's settings page and configured it properly? It's creating those based on its configuration settings.
    – Otto
    Sep 7 '12 at 16:21
  • Are you sure it's the plugin? I use wordpress SEO and it definitely does not add a noindex, follow by default. Does the full meta tag look like this? <meta name='robots' content='noindex,nofollow' /> That can get generated by the Settings >> Privacy screen in the wordpress dashboard. Sep 7 '12 at 16:38
  • Guys, I think I am sure the plugin does this. I have checked Privacy screen, and I am also sure I have configured it properly. This is shown only in domain/page/2/, page/3/, etc. A quick search revealed that this is, indeed, added by default: wordpress.org/support/topic/… I'd really appreciate if you can help! :)
    – Sid
    Sep 7 '12 at 17:13

Ooohhh you mean "pagination" pages. Gotcha now!

It's hard to say for sure(I know I know), but at first glance it looks like this block of code would be what you're after:

if ( $wp_query->query_vars['paged'] && $wp_query->query_vars['paged'] > 1 && isset( $options['noindex-subpages'] ) && $options['noindex-subpages'] ) {
                $robots['index']  = 'noindex';
                $robots['follow'] = 'follow';

Within the pages "Wordpress SEO by Yoast" meta box, open the "Advanced" tab and set the "Meta Robots Index". It took me forever to find this as well.


Although this appeared in the Yoast SEO plugin if I viewed source on my site, I found the problem was simply in Wordpress. Settings > Reading > Search Engine Visibility checkbox was ticked next to "Discourage Search Engines from indexing this site"

Unticking this box, removed the "noindex,follow" from the Code within the Yoast Wordpress SEO plugin & fixed all indexing issues.

Also was responsible for the imaginary robots.txt file disallowing everything that I couldn't find. Hope this helps someone else.

Not the answer you're looking for? Browse other questions tagged or ask your own question.