0

I don't know why, but Google started indexing some pages that we don't have on our server. Mostly pages with the ?tag=22&page=30 query string. By the way, we don't have any "tag", and as result number of 404 pages increased on the site's Google Search Console.

?tag=22&page=30
?tag=27&page=530
?tag=27&page=600
?tag=309&page=81

How can I close/disallow access to this files?

I've added this line, but I'm not sure if it will work:

Disallow: /?tag*
Improve this question
2
  • Please improve your question title. Summarize your specific problem, do not include the word WordPress.
    – fuxia ♦
    Sep 21, 2016 at 19:33
  • Downvoted and close voted as this is just a generic robot.txt question with nothing specific to wordpress in it. anyway don't look at your logs and you will live longer and happier. just because someone put some crap url somewhere do not mean you need to spend time "fixing" it. returning 404 is exactly what should be done and nothing more
    – Mark Kaplun
    Sep 22, 2016 at 3:12

1 Answer 1

Reset to default
0

Here is some info about robots.txt. It looks like your example wouldn't work correctly with the * but, it looks to me like you'd be ok if you just removed the *.

You can always check the robots.txt at with a validator like this one

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.