• I just completed my wordpress website and added it to the Google Webmasters. Verification is done succcessfully and the robot.txt is also submitted in webmasters.

    I am using “yoast seo” pluigin to setup the SEO part. So, I am using their sitemap as of now. But, when I test the sitemap whose path is domain/sitemap_index.xml the google webmaster is giving this error “Url blocked by robots.txt” and there are total 23 URL’s blocked by google.

    I have done following things to rectify the possible issues:

    1. The “discourage search engines” option is not checked.

    2. I have installed a plugin to edit the default robots.txt and my current robots.txt is below:

    User-Agent: *

    Disallow:

    3. I have also checked .htaccess file and it is looking fine.

    I am completely clueless about this issue and how to resolve it. I have deleted my website from webmasters 7-8 hours back and added new link without www. I haven’t submitted the sitemap till now because I hope that I will be able to get some guidance before submission so that this time there will be no error. I still do the test run by giving the sitemap path, but it comes with the same error of link blocking.

    Looking forward to some help..

Viewing 2 replies - 1 through 2 (of 2 total)
  • Moderator Steven Stern (sterndata)

    (@sterndata)

    Volunteer Forum Moderator

    After you change robots.txt, it may take Google several days before it acknowledges the change. Be patient.

    Thread Starter jatin11

    (@jatin11)

    @sterndata – But, I am not able to see the website in search engine at all. When I search by website domain in google, the result comes with error discription and reason mentioned is robots.txt even in search results.

    Indexing will also begin after google acknowledges the change? Till that time, I will not get the links indexed?

Viewing 2 replies - 1 through 2 (of 2 total)
  • The topic ‘Google webmaster is blocking links due to robot.txt in wordpress’ is closed to new replies.