Why is Google Search Console getting X-Robots-Tag noindex?

Question

As can be safely inferred from the question, I'm trying to get Google to index my site.

I'm using Wordpress 5.9 served by nginx (1.13.12), with SEOPress (5.6 up-to-date).

I let the wizard guide me for the settings and created a sitemap that I registered with Google Search Console - as advised.

I let Google do its work and came back later to check, and found that every page exposed in my sitemap returns a noindex meta (X-Robots-Tag with User-Agent 'Googlebot for smartphones').

I did uncheck the "discourage search engines..." checkbox in Wordpress Reading settings.

I also did reset opcache, and I don't have a wp cache plugin (I don't have that much traffic so far).

And when I load the pages in a browser or with curl the X-Robots-Tag is set to none...

$ curl -I https://www.alchimie-web.com
HTTP/1.1 200 OK
Server: nginx/1.13.12
Date: Tue, 26 Apr 2022 15:07:44 GMT
Content-Type: text/html; charset=UTF-8
Connection: keep-alive
X-Powered-By: PHP/7.3.16
Set-Cookie: __wpdm_client=ce7f05b428fa7da70dc3dd9192fdc0a5; secure; HttpOnly
Link: <https://www.alchimie-web.com/wp-json/>; rel="https://api.w.org/"
Link: <https://www.alchimie-web.com/wp-json/wp/v2/pages/7>; rel="alternate"; type="application/json"
Link: <https://www.alchimie-web.com/>; rel=shortlink
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Robots-Tag: none
X-Download-Options: noopen
X-Permitted-Cross-Domain-Policies: none

even with various User-Agents to try and reproduce what the Googlebot reads :

$ curl -A 'Googlebot/2.1 (+http://www.google.com/bot.html)' -I https://www.alchimie-web.com

or

curl -A 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)' -I https://www.alchimie-web.com

or

curl -A 'Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.93 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)' -I https://www.alchimie-web.com

The site never returns a X-Robots-Tag: noindex...

Any hint on what I'm doing wrong ?

Should I use another SEO plugin ? (though I read that the same problem sometimes occurs with Yoast...)

Answer 1

I have a nginx directive that set X-Robots-Tag to none for every page. I thought it was some kind of null value for the header...

Now that I took some time to RTFM, I learned that none means both noindex and nofollow... So it wouldn't let the SEO plugin do its job.

Now that I removed it, it certainly works better.