I found in my tag the following piece of code.

<meta name="robots" content="noindex,follow">

As far as I'm aware this will make sure that robots don't crawl my website, but I want them to crawl my website, how can I change this?

Kind Regards

  • Robots do crawl your site because they follow links (follow), but they don't index it (noindex). – Max Yudin Dec 8 '17 at 13:40
  • Oh oke, thanks for clearing that. Now I need to find how to make sure they index it to. Otherwise my website wont show up on google etc decently. – Arne De Belser Dec 8 '17 at 13:45
  • Check both Settings → Reading and SEO plugin settings, if you use one. – Max Yudin Dec 8 '17 at 14:16
  • As for the reading pane, I think I got that covered. look at this: i.gyazo.com/144295a9620ccff3675c218f5b433cad.png but regarding plugin, I use yoast seo, maybe I have to check that – Arne De Belser Dec 8 '17 at 14:22
  • Keep in mind that Yoast's robots settings can come out to be on the per-post basis. – Max Yudin Dec 8 '17 at 14:33

The actual name depends on your locale but it's under settings => reading and should be called something like "prevent search engines from indexing my site".

| improve this answer | |
  • Yes, I know that. You're referring to this option? i.gyazo.com/144295a9620ccff3675c218f5b433cad.png Sorry it's in Dutch. The reason is, that was already "off", so that's why I don't really know where that line of code is coming from. – Arne De Belser Dec 8 '17 at 13:38
  • Yes, that's what I was talking about. I'd a) check whether it's hardcoded in the theme's header.php, b) search the theme's functions.php and c) look at any plugin that does SEO-related stuff. – janh Dec 8 '17 at 14:09
  • Alright, I'll give it a go – Arne De Belser Dec 8 '17 at 14:22
  • Yes, if your sitewide setting is to allow robots, then it's either a plugin or a theme that's adding this code. – WebElaine Dec 8 '17 at 17:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.