How to allow a specific bot (wget) access

wget is explicitly blocked in your root .htaccess file in these BPS security filters below. you can remove/delete wget from these filters, but the better/safer approach would be to use lynx -source instead of wget -O for CRON commands. If the plugin does not allow you to change the command or use different commands then you will just have to remove wget from the security filers in your root .htaccess file.

RewriteCond %{HTTP_USER_AGENT} (havij|libwww-perl|wget|python|nikto|curl|scan|java|winhttp|clshttp|loader) [NC,OR]

RewriteCond %{HTTP_USER_AGENT} (;|<|>|'|"|\)|\(|%0A|%0D|%22|%27|%28|%3C|%3E|%00).*(libwww-perl|wget|python|nikto|curl|scan|java|winhttp|HTTrack|clshttp|archiver|loader|email|harvest|extract|grab|miner) [NC,OR]

Please post a status update or if the issue/problem is resolved then resolve this Thread. Thanks.

Please post a status update or if the issue/problem is resolved then resolve this Thread. Thanks.

Please post a status update or if the issue/problem is resolved then resolve this Thread. Thanks.

Resolving thread due to lack of response. If the problem is still occuring please reopen/unresolve this thread and post a status update. Thank you.

Sorry for the delay. Finally got the client to remove wget from the .htaccess filters but the page still returns a 404 error. I also noticed that when using a web browser the header also returns a 404 error, but a page does render.

lynx is not a viable solution because of large post payloads that are being sent with wget.

Status Complete
Response Code 404 Not Found
Protocol HTTP/1.1
Method GET
Content-Type text/html; charset=UTF-8

A 404 error usually indicates that the site has a permalink structure problem or some other URL/linking problem.

Which web host is this? Does this host use cPanel?
What is the custom permalink structure? Example: /%postname%/
Have you put BPS in Default Mode to eliminate/verify that BPS is causing this issue/problem?

1. Make a backup of your .htaccess files using BulletProof Security built-in Backup.
2. Activate Default Mode on the Security Modes page.
3. Use the Delete wp-admin .htaccess feature on the Security Modes page.
4. Test your plugin or theme.
5. Restore your .htaccess files using BulletProof Security built-in Restore.

To completely uninstall BulletProof Security you would do steps 2 and 3 above and then just delete the BulletProof Security plugin on the WP Plugins page.

Also since you said you added a skip rule for this plugin. Please post that skip rule.

Host does not use cPanel, it’s a Debian Distro.

Everything worked before client installed BPS (at least had no issues with wget script uploading into file.

Here’s the skip rule

#Risk Survey Spreadsheet Skip
RewriteCond %{REQUEST_URI} ^/members/wp-content/plugins/Biz-risk-survey/ [NC]
RewriteRule . - [S=13]

There is no permalink for this particular url. It is a custom script designed to executed directly with minimal tie-in to wordpress. I can provide more information via a less public venue. If you need more information, please email me at [email protected]

So what happened after you put BPS in Default Mode? Did the custom plugin/script work?

When you say “There is no permalink for this particular url.” I do not really understand what you are saying. A plugin folder or plugin file would not have a permalink. Permalinks are for Posts and Pages. Maybe you mean URI?

If you have a custom script and lets say it is named example.php then you can just do something like this.

RewriteCond %{REQUEST_URI} (example\.php) [NC]
RewriteRule . - [S=13]

But since you are not getting a 403 Forbidden error then a skip/bypass rule is probably not going to be what you need. You need to find out why the URL is not being found – 404 Not Found error.