Over 300 hits from Inktomi/Yahoo User Agent in 1 hr

Sometimes bots go nuts – just block it temporarily and maybe write to Yahoo asking what the deal is here. Most search traffic comes from Google rather than Yahoo so it shouldn’t hurt you too much.

Maybe the bot was spidering all your pages/links? Can you see in your logs which pages have been visited?

Unless you think the world needs your blog I would suggest to block all robots.

Since I have search engines crawling me I have a bit less traffic – but also no spam etc.

I prefer to have readers for a reason – not just random hits just because I have mention sex, boobies or some star in my blog.

Anonymous–I’ll go ahead and block them for now and see what happens. I’ll unblock them in a day and if they come back, then I’ll contact Yahoo. I’ve gotten nearly 500 hits from them in the past 6 hours–this is absurd. Although now that I notice it, Googlebot’s hit my site about 150 times in the same amount of time. Seems excessive to spider a site multiple times a day.

Orangeguru–It’s not spidering my entire site, I don’t think. At the top of my stattraq it says that most of my hits are “multiple posts” which I assume means the main page. Then I have 20 pages that all have 3-5 hits, which isn’t excessive, so I think it might be following a few links, but generally it’s doing the “multiple posts” thing. And alas, I do think the world needs my blog ??

But thanks for both of your advice. We’ll see how it goes.

I’m not sure, but I believe “Multiple posts” doesn’t just mean the front page, it also means all the archive pages.

But this brings up a question of my own. My own installation of WP-Stattraq only has two page type categories: “Mixed” and “Feed.” If I click on a single-post permalink, it still shows up in “Mixed.” I thought it worked that way for everyone, but apparently not. Is this a flaw in Stattraq or in how I’ve configured my site or my permalinks?

I just checked my logs and was hit similarly. But only in the span of an hour. Hardly any data was transfered (50k), and I haven’t been hit since. I don’t think it’s a big deal.

I don’t have problems with excessive hits from Slurp. That honor is reserved for Ask Jeeves/Teoma, which has managed to hit, as near as I can tell, every single page on my site an average of three times this week. It barely registers on the site as a whole, but if you look at just my blog, it chews up 16% of that chunk of bandwidth. Slurp manages about a third of that, Googlebot even less.

No, my problem with Slurp is that they keep hitting long-dead links. You’d think a year’s worth of “301 moved permanently” responses would clue the robot in…