Custom Permalinks Generate Googlebot 404s
-
If I set permalink structure to this…
guitarist.com/blog/year/monthnum/postname/
…Googlebot immediately starts reporting 404 URLs with .html extension added:
guitarist.com/blog/year/monthnum/postname.htmlIf I change the permalinks structure by adding .html…
guitarist.com/blog/year/monthnum/postname.html
…Googlebot immediately starts finding 404s with /blog/ missing:
guitarist.com/year/monthnum/postname.htmlWeird.
Google says both these 404s are from internal links on the blog, but the links are nowhere to be found on the blog itself. They only appear in Google crawl error logs.
With either permalink structure, the blog itself works fine. All internal links connect as they should. No ‘real’ 404s from internal links.
I know I can probably do something with robots.txt or .htaccess to stop Googlebot from reporting these weird 404s, but this problem just started a few weeks ago when I first tried restructuring my permalinks. Before that time, Google reported only a few dozen external 404s per day. Nothing serious. Now I’m getting hundreds of these mysterious internal 404s every day.
I’m told it might be due to a ‘lag’ in Googlebot’s crawling system, and that the problem might clear up in a few weeks or months. But one 404 error disappears and the other one reappears the moment I switch from one permalink structure to the other. So the problem is something more immediate than just a Google lag.
Any ideas?
- The topic ‘Custom Permalinks Generate Googlebot 404s’ is closed to new replies.