Forums
Home / Plugin: Security & Malware scan by CleanTalk / Scrapper questions
(@drachsi)
2 years, 11 months ago
Is it possible to stop scrappers and scanners like screamingfrog from indexing my site?
Regards Drachsi
(@katereji)
Hello. We have the Anti-Crawler option in the Anti-Spam plugin that can help you with this. You can find more details in this article: https://cleantalk.org/help/anti-flood-and-anti-crawler
Hi,
Here is a list of common scrapers. Does it make sense to try to add them?, and would you have a simple example how to do it? https://www.scrapehero.com/top-free-and-paid-web-scraping-tools-and-software/
Regards Pete
(@amagsumov)
Hello Pete.
If you know theirs IP addresses, you can manually add them in your personal blacklist here: https://cleantalk.org/my/show_private?service_type=spamfirewall
And don’t forget to enable the SpamFireWall and Anti-Crawler options in the plugin settings.
Thank you.