Discourage search engines from indexing this site?

  • I just logged into my WP admin site and noticed the developer left the “Discourage search engines from indexing this site” checked. It must have been that way for a long time.

    I’m not super savvy around here but I did a little research and read about the robots.txt file that is created or edited as a result of this parameter being checked. This is what it looks like:

    Site: https://anpreporting.com/robots.txt

    User-agent: *
    Disallow: /wp-admin/
    Allow: /wp-admin/admin-ajax.php

    Is this ok? Or do I need to change something? The “Allow” part is confusing me.

    Thanks in advance so much for your input!

    * edited from original post. I input data and link from the wrong site. Sorry.

Viewing 3 replies - 1 through 3 (of 3 total)

  • This is fine. Allow is default.

    Go to Google Search Console and register the website – it will tell you what to update, change and improve your ranking

    The easy thing to do is not worry about the robots.txt file and just set that setting to ‘off’. That will make any required changes to the robots.txt file to let search engine bots index the site.

    I know this is old and you probably handled it already, but in case you didn’t (or for future readers of this thread) the “Discourage search engines from indexing this site” box should be unchecked – and make sure you click “save” after you uncheck it. Unless you want to try to keep people from finding your website.

Viewing 3 replies - 1 through 3 (of 3 total)
  • The topic ‘Discourage search engines from indexing this site?’ is closed to new replies.