1

on every WordPress installation there is a default virtual robots.txt file with the following

User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php

but on my 2 last project /robots.txt give me a 404 instead.

  • I tried to disable all plugin
  • I tried to switch theme with a blank one
  • Tried to look on another older project which I updated last week and robots.txt is working

This not server related because it work on my local environment with older project

Any idea why ?

Improve this question
1
  • I have no idea why it is missing but you could just upload a robots.txt via ftp.
    – Luckyfella
    Commented Aug 1, 2017 at 8:11

1 Answer 1

Reset to default
0

Nevermind. It just doesn't work when wordpress is in a subfolder.

(robots.txt should be on website root so just create your own if wordpress is in a subfolder)

Improve this answer
3
  • This does not provide an answer to the question. To critique or request clarification from an author, leave a comment below their post. - From Review
    – cjbj
    Commented Aug 1, 2017 at 8:46
  • 1
    It does answer question. Just try before downvoting... seriously
    – David Horizonef
    Commented Aug 1, 2017 at 11:37
  • wp-includes/functions.php generates this file dynamically on request, to overite this behavior just create a file manually wordpress.org/support/topic/…
    – bjornredemption
    Commented Jul 16, 2020 at 13:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.