Site crawl error: Scan aborts and yellow “Scan Stage Failed” banner appears

Good afternoon.

We have been using your Wordfence plugin for a long time and successfully because it has features that other security plugins do not have.

Unfortunately, some time ago, the following functions stopped working on our site:

• Site crawl error: Scan aborts and yellow “Scan Stage Failed” banner appears
• the 2FA function stopped working: you can enter any code and access the site

We ran diagnostics and found the following:

1. On the “Tools”/”Diagnostics” page, in the “Connecting back to this site” section, a message is displayed:
   wp_remote_post() test back to this server failed! Response was: 200 OK

This additional info may help you diagnose the issue. The response headers we received were:

HTTP/1.1 200 OK
Connection: close
content-type: text/html
x-litespeed-tag: c05_
content-length: 213
content-encoding: gzip
vary: Accept-Encoding
date: Wed, 26 Apr 2023 07:26:14 GMT
server: LiteSpeed
x-qc-pop: EU-PL-WAW-349
alt-svc: h3=”:443″; ma=2592000, h3-29=”:443″; ma=2592000, h3-Q050=”:443″; ma=2592000, h3-Q046=”:443″; ma=2592000, h3-Q043=”:443″; ma=2592000, quic=”:443″; ma=2592000; v=”43,46″

2. The “LiteSpeed” caching plugin is installed on the site, but the “Health Check & Troubleshooting” plugin does not see it and displays a message:

Page caching is detected by looking for an active page cache plugin, as well as making three requests to the main page and looking for one or more of the following cache headers for the HTTP client:
cache-control, expires, age, last-modified, etag, x-cache-enabled, x-cache-disabled, x-srcache-store-status, x-srcache-fetch-status, x-litespeed-cache, x-lsadc-cache, x-qc-cache.
The average server response time was 2,549 ms. It should be less than the recommended threshold of 600ms.
No client-side caching headers found in response headers.

Page caching plugin not found.

3. Daily and monthly archives do not work – there is a redirect to the main page of the site (archives by tags and categories work correctly).
4. In the front office of the site, messages “clearing erroneous cookies” and “many redirects” periodically appear.
5. High server CPU load.

We applied other security plugins:

• All In One WP Security 5.1.7,
• WP Cerber Security, Anti-spam & Malware Scan 9.5.4,
• Anti-Malware Security and Brute-Force Firewall 4.21.92
  but they don’t fix the problem.

Therefore, assuming that the problem is on our side (on our site), we did the following:

• In accordance with your recommendations, https://www.wordfence.com/help/scan/troubleshooting/ met all the requirements – this did not solve our problem.
• In accordance with the recommendations of WordPress, deactivated all plugins – this did not solve our problem.
• since the site is running on a shared hosting plan, we contacted the hosting company and were told that all restrictions were removed from our site – this did not solve our problem.

As a result of the analysis of this problem with the specialists of the hosting company, we have 2 options for solving it:

• if Wordfence uses User-Agent during external scanning of the site and it is blocked on the hosting, then we need the name of this User-Agent so that the hosting excludes it from blocking
• if the reason is that the .htaccess file from the uninstalled plugin remains in some directory (since the site is used for development, different plugins are tested on it, it works in the “Multisite” mode and more than 130 plugins are installed on it), then can you advise with what parameters to run a site crawl so that Wordfence detects the .htaccess file that breaks the site.

We have 2 questions:

• if Wordfence uses User-Agent, then we need to know its name
• if the reason is in the .htaccess file, then we need to know with what parameters to run Wordfence in order to find this file.

Server Options:

• WordPress 6.2
• PHP 7.4.33
• Apache/2.4.51 (Unix)
• MySQL 5.7.21-20-beget-5.7.21-20-1-log

I hope that you can answer these 2 questions and help in solving this problem.

Thank you.