Plugin can’t block bingbot

  • Resolved jenny2222

    (@jenny2222)


    3 years, 3 months ago

    Hello, I want to block bingbot from my site. E.g. I have many 404 pages (in the past, but deleted already), which bingbot still keep crawling, resulting my server to be maximised.

    I removed bingbot under goodbot and added bingbot to the bad bot list. But I still see many bingbots crawling my 404 pages.

    Any help on this?

Viewing 5 replies - 1 through 5 (of 5 total)
  • Plugin Author Jeff Starr

    (@specialk)

    Glad to help. To help me understand, can you explain how you did this:

    “I removed bingbot under goodbot and added bingbot to the bad bot list”

    The free version of Blackhole does not enable manually adding bots, so I’m wondering if you are using the pro version..?

    Thread Starter jenny2222

    (@jenny2222)

    apologies, i saw wrong, i removed bingbot, but did not add bingbot to bad bot list.

    Is this implied that it would stop bingbot? or only pro version cant be done?

    Plugin Author Jeff Starr

    (@specialk)

    Thanks for explaining.

    The plugin works by blocking any bot that disobeys the site’s robots/nofollow rules and follows the hidden trigger link. This applies to any bot that is not explicitly allowed via the settings. So with that in mind, if bingbot disobeys robots rules and falls into the trap, it will be blocked. Otherwise, if bingbot does not disobey and fall into the trap, it will not be blocked.

    I hope this helps. Let me know if I can provide any further infos.

    Thread Starter jenny2222

    (@jenny2222)

    I see, so how do I make bingbot disobey the rules where i don’t want them to crawl non-existence site, would that be possible?

    Or i would need to make them disobey existing site first, thus result in all pages,even non-existence one? Thanks

    Plugin Author Jeff Starr

    (@specialk)

    Glad to help:

    1) “how do I make bingbot disobey the rules where i don’t want them to crawl non-existence site, would that be possible?”

    It’s not possible with the free version (it’s a pro feature).

    2) “would need to make them disobey existing site first, thus result in all pages,even non-existence one?”

    Bad bots are blocked from all site access, doesn’t matter if the page exists or not (404).

Viewing 5 replies - 1 through 5 (of 5 total)
  • The topic ‘Plugin can’t block bingbot’ is closed to new replies.