pc robots.txt – site delisted

  • Resolved ds1970

    (@ds1970)


    13 years, 8 months ago

    Hi, something strange happened and I don’t know if it’s directly related to me installing the pc robots.txt plugin a couple of days ago.

    My site was on page 3 of google last week for its topic, but today, it’s no longer in the search results on any page.

    Could the pc robots.txt file affect if a site gets delisted or deindexed? installing this plugin is the only thing I’ve done with the site in the past few days, so I’m confused as to what happened. I didn’t change any of its setting, so I’m at a loss.

    thanks

Viewing 5 replies - 1 through 5 (of 5 total)

  • Could the pc robots.txt file affect if a site gets delisted or deindexed?

    Absolutely. Not sure if you understand what robots.txt are for, but if improperly configured, it could be telling Google or other search engines not to spider your site.

    Can you provide a link to your site so I can take a look at your robots.txt?

    Thread Starter ds1970

    (@ds1970)

    Hi, thanks for your response.
    Here’s the link: https://www.breast-pump-reviews.net/robots.txt

    what’s inside is what the plugin created.

    thanks

    Thread Starter ds1970

    (@ds1970)

    Also, I unchecked the box for “Add sitemap URL to the virtual robots.txt file” in my xml sitemaps today. i read that it needs to be unchecked after you install pc robots.txt. Is that correct? thanks

    I personally would remove everything except these lines:

    User-agent: *
Disallow: /cgi-bin/
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /wp-content/plugins/
Disallow: /wp-content/cache/
Disallow: /wp-content/themes/
Disallow: /wp-login.php
Disallow: /wp-register.php

    Also, I unchecked the box for “Add sitemap URL to the virtual robots.txt file” in my xml sitemaps today. i read that it needs to be unchecked after you install pc robots.txt. Is that correct?

    Sorry, I don’t know.

    You should not have to specify a path to your sitemap in robots.txt. As long as it’s properly named (sitemap.xml) and at the root of your site, spiders should be able to find it.

    Thread Starter ds1970

    (@ds1970)

    Cool, thanks for your help.

Viewing 5 replies - 1 through 5 (of 5 total)
  • The topic ‘pc robots.txt – site delisted’ is closed to new replies.