wf_hoover table huge and scan termination errors after plugin upgrade

Thank you for the detailed report, and for referencing the similar issue with a link.

Can you try turning on the option “Enable debugging mode (increases database load)” on the Wordfence Options page, then running a manual scan? Once the scan stops, probably at the same point it stopped above, click the link “Email activity log” above the box, and enter my email address:
mattr (at) wordfence.com

There may be a limit at your host on the size of data transferred to/from other sites for outbound connections, so this may help narrow down the issue.

It’s probably good that it is checking your cache folder for bad links, as some malware is good at hiding links in code, but they may still appear in the cached files. You could exclude the backup log files using the option “Exclude files from scan that match these wildcard patterns” (under the “Scans to Include” section on the Wordfence Options page) — that may buy some time if it is a size limit at the host, but the problem would likely come back again, as the site grows.

Thanks, Matt. I just sent the activity log (with debug on). I have not yet changed the file exclude settings.

I should also mention that I’m on a healthy VPS and size limits don’t seem to be a problem, as they’ve previously been on shared hosts.

I’ll look forward to your feedback.

Thanks!

Thanks for sending the activity log. It has a huge number of errors from W3 Total Cache — can you try temporarily disabling the W3TC plugin, and try running a scan after that?

If it does work that way, you can try a scan again with W3TC enabled, but I would suspect the database cache first, and disable just that feature.

A long time ago, I did have issues with the database cache interfering with other plugins I was using. I don’t know for sure if this is the problem, so if it doesn’t help, let me know.

It seems like something has been fixed, or at least something has changed. (Note that I have not made any modifications on my end.)

I notice that scans now fill the wf_hoover table…with 100s of MBs. However, the wf_hoover table seems to be emptied or truncated to 0MB upon completion of the scan.

I just thought I’d ask: is this the way Wordfence has always worked, but I just haven’t noticed?

Thanks!

Yes, Wordfence always adds a lot of temporary information to wp_wfHoover (the amount varies depending on your site’s content), and usually the table is truncated at the end of a scan. If the scan fails, sometimes the temporary data can stay in the table instead.

I had one other person with a similar problem yesterday, and turning off the database cache in W3TC seemed to help. On your site, if you haven’t done that yet, it might be that whatever data was cached didn’t happen to be in the cache at the time the scan started — so it might fail again randomly.

You could let it ride and see if the problem comes back, or you could turn off the database cache in case that was the original issue. I believe the other parts of W3TC should still work fine — I used to work on a site where W3TC’s database cache caused problems with certain other plugins, but we didn’t have problems with any of the features.

Thanks. I’d like to keep database caching on, as it has been recommended for my site configuration by multiple people I’ve worked with and is the result of a bit of tweaking over the years to determine what works.

I’ll keep an eye on W3TC vs. WF. The odd thing is that my site ran without a hitch since adopting Wordfence — only until very recently. So I just wonder what variable changed to cause the scan failure and heaps of data being piled up into the database. Just seems odd.

If it happens again, I’ll open a new thread or re-open this one. Thanks for your time and attention.