Constant Head Requests by Google Bot

Oops, I just saw a sticky about this. Let me read that first.

Jose

BPS will not negatively affect your SERP’s / ranking or block Google from crawling and indexing your site. What we are aware of is that if the bot really is the Googlebot and not a hacker or spammer faking that they are the Googlebot and an image file is being retrieved then several different things are occurring besides image retrieval. Whatever those other different things are will get blocked by BPS. This does not block your images from being retrieved by Google or website visitors or anything else that is important.

I am pretty sure that Google does not make HEAD Requests so the scenario is probably that one of your plugins is making HEAD Requests and Google is somehow involved in that equation or this is a hacker or spammer faking that they are the Googlebot.
https://support.google.com/feedburner/answer/79023?hl=en

You can check your URL’s from your Google Webmaster account if you want to verify that there are no problems, but we get this same question year after year and BPS does not block the Googlebot or any other search engines from doing what they need to do.

Also the Broken Link Checker plugin tries a HEAD Request and if that does not work then it uses alternative / fallback code. So you will see log entries for this, but BPS does not block the Broken Link Checker plugin from doing what it needs to do.

Thanks.

Regarding the link, the article implies that we should have head requests on. I have a podcast actually, and head is taken off, and the podcast also uses feedburner for the feed and then iTunes picks up on it via feedburner.

Everything is working fine.

Before it was actually a problem. I submitted the actual podcast feed to iTunes and not via feedburner and the head requests were a problem. I had to redirect that feed to feedburner eventually. With a new podcast, I just went via feedburner.

Anyhow regarding Google bots, I also read the sticky in the support forums. I understood some of it.

The latest “google bot” to hit the site was:

HTTP_USER_AGENT: Googlebot-Video/1.0

It will take some time for me to understand these things. I’ll check Webmasters to see if everything looks good.

thank you,

jose

Regarding the Broken Link Checker…

That is good.

Though, lately it has been giving “unknown errors” and saying links are broken when they are not. It was happening every now and then, but now it is a continuing problem. I can’t trust it anymore when it says i have x number of broken links because they actually aren’t.

I wasn’t sure if this was an issue with the plugin, or now that I think about it, an issue with BPS configurations?

Jose

In Google Webmaster tools use the link checker tool (something like check as Google) and enter the URL that you see in the BPS Security Log entry to check the URL.

Yep, that is a known issue with BLC and there is a known solution out there somewhere. Google that issue/problem and the solution has something to do with using the advanced options in BLC. It has been at least a year since I was last asked this question so it has been deleted from my brain except for the excerpt. ??

Thanks for the Google tip.

I will look into that BLC prob.

Thanks,

Jose

Resolved.