How to block content scraping bots?

  • Resolved marbs

    (@marbs)


    3 years ago

    Hello, I am using your plugin Defender.

    In the User Agent Banning section I have added the following:

    archive.org_bot
    archive.org
    archive
    ia_archiver
    archive.is
    But it seems it’s not blocking them out.

    The reason I want to do this is because privacy protection of my community members and copyright violation.

    I asked them by email kindly to exclude my site but it comes down to they don’t want to cooperate.

    So I hope your plugin can help my site protect from content scraping bots.

Viewing 2 replies - 1 through 2 (of 2 total)
  • Plugin Support Patrick – WPMU DEV Support

    (@wpmudevsupport12)

    Hi @marbs

    Sorry to hear you are having this issue.

    Could you double-check if you flushed the caching after adding the rules?

    Sometimes a caching plugin / hosting will render the cached file before getting the lockout page.

    I added those UA to my Defender and tested using my browser and got blocked: https://www.searchenginejournal.com/change-user-agent/368448/ Can you test it on your site using the browser too?

    Best Regards
    Patrick Freitas

    Plugin Support Patrick – WPMU DEV Support

    (@wpmudevsupport12)

    Hi @marbs

    I hope you are doing well and safe!

    We haven’t heard from you in a while, I’ll mark this thread as resolved.

    Feel free to let us know if you have any additional questions or problems.

    Best Regards
    Patrick Freitas

Viewing 2 replies - 1 through 2 (of 2 total)
  • The topic ‘How to block content scraping bots?’ is closed to new replies.