Whitelist known Crawl Bots

  • Is there a way we can whitelist known Bots like google,bing,yahoo,MSN? I could see in the log that when google bots try to crawl the website, they get blocked being listed as VPN/Proxy. It really has a negative effect. As of now i am trying to manually whitelist all IPs but it is a very unreliable as the IPS are different everytime. Cloudflare has a whitelist to allow known bots. If you could implement the same, it would be great!

Viewing 1 replies (of 1 total)
  • Plugin Author Proxy & VPN Blocker

    (@rickstermuk)

    Hi @darshil5p

    This shouldn’t be happening at all, are you using the Country/Continent Blacklist or Whitelist by chance?

    The proxycheck.io API itself doesn’t detect these bots so there is some other issue going on here.

    Thank you,
    Rick.

Viewing 1 replies (of 1 total)
  • The topic ‘Whitelist known Crawl Bots’ is closed to new replies.