Fake Google crawlers Identification – ipv6 mapping problem?

Can you tell me what hosting provider you are using, and whether you use any type of reverse proxy? I haven’t seen IPv4-mapped addresses appear like this before on hosts that have IPv6 enabled.

Also, are you using any type of proxy service (like CloudFlare) or reverse proxy on the host? These generally should be ok to use, but may have a configuration we have not seen yet.

Of course, I recommend disabling “Immediately block fake Google crawlers” for now.

-Matt R

I have a Dedicated Server at 1&1 as Hosting Provider. The server is running Debian and nginx. I am not using a reverse proxy and no proxy service.

I disabled “Immediately block fake Google Crawlers” when I saw that it couldn’t identify Google.

That might be it — nginx can convert IPV4 addresses to this format, depending on the “ipv6only” option in your listen directive(s). I found a post about it here:
https://serverfault.com/questions/638367/do-you-need-separate-ipv4-and-ipv6-listen-directives-in-nginx

If that helps, let me know. We could possibly work around this in a future version, but changing the nginx config could help in the short term, if it works for your setup.

-Matt R

I made the change in the nginx configuration like suggested in the post you have linked to.

I replaced:

listen [::]:80 ipv6only=off default_server;

with:

listen 80;
listen [::]:80;

Than I restarted nginx and activated:
“Immediately block fake Google crawlers”

Instead of getting this:
Wordfence Live Activity: Blocking fake Googlebot at IP ::ffff:66.249.64.51

I get now this:
Wordfence Live Activity: Blocking fake Googlebot at IP 66.249.64.51

Seems like Wordfence is still blocking a real Googlebot even now it’s seeing the IP Address in IPV4 Format.

I switched off the blocking feature for now, but wanted to give you feedback on this.

Thanks for the additional details. There is an internal cache that may be holding the previous failed verification, from when the IP address came through in the IPv4-mapped IPv6 format — I think that is the most likely cause, now that the IPv4 address appears normally.

If you want to try clearing that cache, you can either reinstall Wordfence with the “Delete Wordfence tables and data on deactivation” option enabled (but you would have to re-enter your Wordfence settings), or truncate the table “wp_wfCrawlers” in the mysql database, if you’re comfortable enough with mysql — so that the lookups will be done upon the next hit.

After that cache is cleared, if you have the Live Traffic view enabled, you can check the “Google Crawlers” tab to see if future visits are identified correctly, even if you don’t re-enable the “Immediately block fake Google crawlers” option yet.

If you don’t want to do that, the cache will still clear itself in a maximum of 7 days.

If you try it, let me know how it goes. Either way, I’ll put in a case for our dev team to check this out. It’s likely that we could support this format of address in a future version.

-Matt R

I truncated the table “wp_wfCrawlers” and than checked “Google Crawlers”. Looked good. After that I activated “Immediately block fake Google crawlers”. Seems that now it is working fine.

Thank you!

Great, thanks for the response. I’m glad it’s working again, and all the details helped narrow down the cause. I’ve entered this for our dev team to check it out (internal reference number FB1317), so that we can support either configuration, in a future version.

-Matt R