Virtual robots.txt wont go away

  • Hi!

    Today I realised that my new site wasnt being crawled because my privacy setting were switched on.

    I disabled privacy but google webmaster tools still says that I have a robot.txt reporting

    User-agent: *
    Disallow: /

    Sitemap: https://pathofmostresistance.org/sitemap.xml.gz

    So I created my own robot.txt file and put it in the root directory (public_html)but google still came back with the same error.

    I installed the Robots.txt plugin and editied the file from that and still get the same problem. When I visit

    It looks fine. I suspect that the virtual robots.txt file is somehow taking precedent over the one in my root directory. Somehow Googles bots are getting redirected their is my only guess!

    Im stuck!

Viewing 2 replies - 1 through 2 (of 2 total)

  • Currently my site isn’t being indexed by google, I receive the following message from google webmasters tools:

    “The page could not be crawled because it is blocked by robots.txt.”

    I’ve read a virtual robots.txt is generated in wordpress but when I check my site.com/robots.txt it shows “Disallow: /”

    So far I made sure my Blog Visibility under Private Settings is set to “visible to everyone, including search engines”

    No changes.

    I made a new robots.txt with “User-agent: * Allow: /”, uploaded to my root.

    No changes.

    What other options am I left with? Should I deactivate XML Sitemap Generator for WordPress 3.1.2? I don’t want to install a robots.txt plugin.

    Did you try deleting the robots.txt file?

Viewing 2 replies - 1 through 2 (of 2 total)
  • The topic ‘Virtual robots.txt wont go away’ is closed to new replies.