Blocking bad bots in htaccess and Wordfence

I haven’t had any problems using Wordfence and additional .htaccess rules to block bad user agents. That Bot Reports list is mighty long. What’s your approach?

See https://www.allthingsdemocrat.com/htaccess-security-and-the-largest-block-bots-list-on-the-web/ ??

Thanks for the link. Great article, and fascinating to read of the lack of impact on performance. I’ve got the 6G Blacklist running with Wordfence, and have had no issues at all. Let me know how you get on with adding your exclusion rules ??

Thanks for the heads-up on 6G. I didn’t realize that existed. The 2014 Micro Blacklist is two years newer than the 6G Beta, which is odd. I’ll have to go through the comments after that post – I see on the first page that some had issues with the code. However, I wasn’t aware that the blank user agent block was an issue so I updated my htaccess and my post and text file.

Thanks!

Yep, it’s a pretty aggressive firewall, so it’s definitely a good idea to check the comments first, and then work through the site after enabling of course (I had issue with the thumbnail on my blog page which were being blocked by thumb|timthumb| in the request strings, and have had to comment out RewriteCond %{QUERY_STRING} ([a-zA-Z0-9]{32}) [NC,OR] to access and download my backups). I think this is why it’s still in beta, as it really needs to be tailored to each setup, rather than the 5G which worked out of the box.

Cheers!

I am struggling to stop the teleport pro bot but when I add a line to block it my site will get a 403 redirect path warning by Ayima addon in the chrome browser. Anyone knows how to prevent this? I tried to eliminate the second rewrite rule but that lifted the block and with the downloaded version of TP Pro I was able to “scrape” the site again.

# BEGIN WordPress

<IfModule mod_rewrite.c>

RewriteEngine On

RewriteBase /

RewriteRule ^index\.php$ - [L]

RewriteCond %{REQUEST_FILENAME} !-f

RewriteCond %{REQUEST_FILENAME} !-d

RewriteRule . /index.php [L]

</IfModule>

RewriteEngine On 

RewriteCond %{HTTP_USER_AGENT} ^Teleport\ Pro [OR]

RewriteRule ^.* - [F,L]