Locked out Google?

Hi there Alfred,

Thank you for being one of early users, especially as one of the users that work on improving our plugin with your interactions.

We did confirm one of Google’s crawl bot addresses (66.249.66.86) as marked “offender” in bad_ip public database. Our plugin detected query /weblog/page/9/?id=1&jsn_setmobile=yes that resulted with 404 as bad query.

As bad_ip work with global cloud database, IP’s are checked against whole bad_ip’s public database, and sometimes IP that you see in Reporting section wont be actually blocked. To remove the address you would have to visit https://bad-ip.info/, search for given IP, and if the address is there, you will have section saying “If you can confirm resolving of the problem that involved this IP in the incident(s), you can send request to our service for removal from the database” with button which will lead you to the form where you can request link for unlisting the IP from bad_ip database sent to your email.

We will always work to improve backend algorithms for detecting false positives.

Sections for manual adding of IP’s to be black or white listed will be released in next version. Link to automatically submit unlist request from your plugin page will be added in next version too.

This situation with Google implied to a new settings that will be implemented and will deal with crawl bots based on users choice.

We did unlist given IP as confirmed false positive and will try to deliver updated version of plugin as soon as possible.

Thank you again

thanks for the comprehensive answer, I look forward to the new version.There are still a lot of Google adresses in the list and they look like they are made by direct calls to js and so on in the plugins directorie.

• This reply was modified 4 years, 10 months ago by AlfredG.

Hi there Alfred,

We have just released new version of the plugin adding several segments, including switch checker for crawler bots and manual white and black lists under settings page and ability to check more info or send unlist request on IP from main plugin page inside “Denied Access” section.

Yes, there is a number of crawler bots addresses still inside our public database recorded as “offenders”. Most of them will be cleared as recorded incidents get manual review and prove as false positives. As you are probably aware, most of “bad actors” can easily set their user agent to look like known crawler bots, so we cannot implement global settings on our backend to allow them, but now we gave users a choice to allow them on their side and that will override all reporting and blocking mechanisms even if they are recorded in our database. Beside that, now when we working on revamping this project, we will appoint some of our SecOps personnel to manually review our public database on regular basis.

Again, thank you for getting attention to this matter.

Hope you will continue using bad_ip and that it will serve you well

Regards

Thanks, I installen it is a major impovement You made.