Index.php “Indexed, though blocked by robots.txt”

  • Resolved billbennett

    (@billbennett)


    4 years, 7 months ago

    I don’t think this is specific to Yoast, but it is related to WordPress and Google Search Console.

    In my coverage report I keep seeing my index.php page https://billbennett.co.nz/wp-admin/index.php

    reported under Indexed, though blocked by robots.txt

    A month ago I submitted a request to remove the page. It dropped out of the Coverage report for about eight days and now it is back.

    1. Is this something I should worry about? I don’t think so, but I’d like an expert opinion.

    2. If it is, can I use Yoast SEO to remove it?

Viewing 5 replies - 1 through 5 (of 5 total)
  • Plugin Support devnihil

    (@devnihil)

    @billbennett This isn’t related to Yoast, but is likely caused by the following lines in your robots.txt:

    User-agent: *
Disallow: /wp-admin/

    We don’t mean to imply that you should modify this as it’s fairly common to have this in the robots.txt to prevent indexing of any of it’s content. We just mean to say that Google crawling the https://billbennett.co.nz/wp-admin/index.php AND the presence of that Disallow directive is why you receive the ‘blocked by robots.txt’ .

    As for how/why Google is crawling the admin.php URL, we can definitely say, but it’s typically due to it finding a link to the page somewhere on your site, and can frequently be caused by a link to the URL being used somewhere in your site’s theme.

    Unfortunately, this isn’t something that you could use Yoast to remove from Google as Yoast can only add a noindex tag to a page made in WordPress, not a .php file that is part of the core WordPress files.

    Thread Starter billbennett

    (@billbennett)

    Thanks. I may try digging around in the theme to see if there is a call to the page.

    I should have mentioned that Search Console doesn’t help here, it says:

    Referring page none detected.

    Do let us know what you find out. We’ll leave this post open for 7 days just in case someone reading the forums may have other inputs/suggestions.

    Plugin Support devnihil

    (@devnihil)

    We are going ahead and marking this issue as resolved due to inactivity. If you require any further assistance please create a new issue.

    Thread Starter billbennett

    (@billbennett)

    Yes. I can’t find a call to the page anywhere in the code, although I’m not sure how I would find it. Searching for the URL gets me nowhere.

Viewing 5 replies - 1 through 5 (of 5 total)
  • The topic ‘Index.php “Indexed, though blocked by robots.txt”’ is closed to new replies.