Google Blocked Page Indexing

  • michlangley

    (@michlangley)


    5 years ago

    After installing wordfence, just realized weeks later google being block. I did have settings to allow google. Tried changing to allow anyone claiming to be google but still google could not crawl. Google was indicating a crawl anomaly. The Googlebot desktop did have Yes marked on Crawl allowed. I had to turn WordFence off and now recrawling every page that was blocked. It seems like a good program for all it did catch but this is crazy. Anyone have success making changes and able to still use wordfence?

Viewing 3 replies - 1 through 3 (of 3 total)
  • WFGerroald

    (@wfgerald)

    Hey @michlangley,

    Can you take a look at Wordfence -> Tools -> Live Traffic to see why the Google bots are being blocked?

    Are there any manual rules being set within Wordfence -> Firewall?

    Please let me know.

    Thanks,

    Gerroald

    Thread Starter michlangley

    (@michlangley)

    @wfgerald I disabled the plugin. And the google was able to crawl. I am nervous about turning back on. I would need to this correct to check? I do not think I had wordfence firewall manual rules. My only thought is I may have excluded anyone not in the US. Appreciate detailed information on how to check and I can quickly turn back on. I was only able to get a non crawl error in google when I turned off plug in.

    Plugin Support wfscott

    (@wfscott)

    @michlangley

    You would need to briefly turn on the plugin to check those, that is correct.

    The first thing I would check would be the Rate Limiting settings in Wordfence > All Options > Rate Limiting. There, be sure you have nothing set below ~30. If you have any of the options set below 30 and are blocking rather than throttling, that may likely be the issue. Here are our recommendations for Rate Limiting: https://www.wordfence.com/help/firewall/rate-limiting/

    Also, as Gerroald mentioned, check your Wordfence > Firewall > Blocking area at the bottom in the Current Blocks section. There make sure you aren’t blocking any ranges that might be Google IPs. You mentioned you may have excluded anyone not in the US. Double-check that Blocking area for any country blocks set up for Europe or the USA.

    Scott

Viewing 3 replies - 1 through 3 (of 3 total)
  • The topic ‘Google Blocked Page Indexing’ is closed to new replies.