Googlebot going crazy

  • Matt Knowles

    (@aestheticdesign)


    5 years, 7 months ago

    Today I noticed this site was suspended as it exceeded it’s monthly bandwidth allocation. Looking at stats, Googlebot was by far the worst offender. Looking at log files I saw numerous entries like this:

    66.249.75.58 – – [09/Apr/2019:15:44:15 -0500] “GET /?cat=-1&page_number_0=2&page_number_4=3&page_number_6=1&page_number_19=2&page_number_20=2&page_number_16=2&page_number_5=2 HTTP/1.1” 200 457785 “-” “Mozilla/5.0 (compatible; Googlebot/2.1; +https://www.google.com/bot.html)”

    I have All In One SEO installed and canonical URL’s enabled. Any idea why Google is accessing the pages using these types of URLs?

    I added an entry to the robots.txt file to disallow /cat=* and hopefully that will take care of it, but I’m wondering what would have triggered this sudden behavior from Googlebot. Also what’s with all the different page number parameters?

    The page I need help with: [log in to see the link]

Viewing 2 replies - 1 through 2 (of 2 total)
Viewing 2 replies - 1 through 2 (of 2 total)
  • The topic ‘Googlebot going crazy’ is closed to new replies.