Memory allocation errors in error log

Hi @mishash,

It appears that in your case 256M is indeed not enough; it depends on your site, resources, and plugins and themes.

Please see this article from our documentation which addresses this question.

Thanks @wfyann. I read the article, but it seems to be a bit extreme for a security plugin to use over 256M, even for big sites. Based on my 20+ years of software experience, when you write a code, you should expect this code to operate within reasonable resources rather than expect unlimited resources. And 256M is more than reasonable… ??

Hi @mishash,

As explained in the article the PHP memory_limit applies to all PHP processes running at given time (not only Wordfence related PHP processes).

For example, if you have 10 plugins and combined they consume more memory than you have allocated, you’re going to encounter this error.

Thank you @wfyann, now I understand. I didn’t realize the limit applies to all PHP processes. Will increase the limit to 512M, hopefully the errors will go away.

@wfyann, I increased memory limit to 512M, restarted Apache service and run manual scan. 5 min into the scan I got the following error:
[MAR 03 18:27:32] Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 1048577 bytes) in /var/www/vhosts/exampe.com/httpdocs/blog/wp-content/plugins/wordfence/lib/wordfenceScanner.php on line 303

There is certainly a serious problem with memory allocation in Wordfence, as the only errors in the log (related to PHP memory) are from Wordfence plugin and, considering that I just doubled the allowed memory to 512M, it is obvious that Wordfence used up extra $256M on it’s own in 5 min of operation.

• This reply was modified 6 years, 8 months ago by mishash.
• This reply was modified 6 years, 8 months ago by mishash.

Well, I checked the “Use low resource scanning” checkbox and… the result is exactly the same, it runs out of memory at exactly the same point. In fact, the only difference in the log between records for the “low resources” block and regular resources bock is one line “Using low resource scanning”, so it looks like this “low resources” checkbox makes no difference at all. I’ve been using Wordfence for over 3 years and never had such problems in previous versions ??

Here is the log with 2 scans (with and without low resources setting): https://justpaste.it/1ht0n

Hi @mishash,

What value have you specified for the “How much memory should Wordfence request when scanning” parameter?

You can check that on the “All Options” page in the “Scan Options –> Performance Options” section.

Hi @wfyann
I tried 3 values: empty (i.e. default), 128Mb, 256Mb. End result is the same: out of memory error. FYI, the site has 118 posts and 43 pages. It is running Weaver Xtreme theme and has 14 plugins installed – see this image for plugin list/impact. As I already mentioned, I am running Wordfence since 2014 and had no issues with scanning until the last version. The site is running on VPS with dedicated CPU core and 2Gb of RAM. According to monitoring software, typical allocated memory rarely exceeds 1Gb, so there should be plenty of memory available.

• This reply was modified 6 years, 8 months ago by mishash.
• This reply was modified 6 years, 8 months ago by mishash.

Hi @mishash,

Sorry about the delayed update.

Could you please run scans with “debugging mode” enabled and check if it’s always failing right after scanning one specific file.

It looks like the issue occurs when the scanner is reading a file in chunks and it’s failing trying to calculate the size of the file.

Can you confirm that it’s always happening with the same file? If so, could you try excluding that file from scan?

Hi @mishash,

Could you also please confirm if you have either of these plugins installed on your site?

Hi @wfyann, I don’t use Query Monitor and W3 Total Cache plugins.

I enabled Debug mode in Diagnostics and tried 3 scans. In these 3 scans the “out of memory” error happened when scanning totally different files. When looking at debug output in log files I can clearly see that allocated memory slowly grows to 510M+ and it never gets released. Please see below the logs from 3 scans prior to the error:

Scan 1
? [Apr 09 13:35:52] Scanning contents: wp-content/plugins/wp-rocket/vendor/composer/installers/src/Composer/Installers/KodiCMSInstaller.php (Size: 219 B Mem: 509.5 MB)
? [Apr 09 13:35:52] Scanning contents: wp-content/plugins/wp-rocket/vendor/composer/installers/src/Composer/Installers/KohanaInstaller.php (Size: 167 B Mem: 510.75 MB)
? [Apr 09 13:35:52] Scanning contents: wp-content/plugins/wp-rocket/vendor/composer/installers/src/Composer/Installers/LanManagementSystemInstaller.php (Size: 726 B Mem: 511.75 MB)
? [Apr 09 13:35:52] Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 1048577 bytes) in /var/www/vhosts/example.com/httpdocs/blog/wp-content/plugins/wordfence/lib/wordfenceScanner.php on line 303

Scan 2
? [Apr 09 13:56:18] Scanning contents: wp-content/plugins/wp-rocket/vendor/composer/installers/tests/bootstrap.php (Size: 111 B Mem: 507.25 MB)
? [Apr 09 13:56:18] Scanning contents: wp-content/plugins/wp-rocket/vendor/jamesryanbell/cloudflare/composer.json (Size: 498 B Mem: 508.25 MB)
? [Apr 09 13:56:18] Scanning contents: wp-content/plugins/wp-rocket/vendor/jamesryanbell/cloudflare/composer.lock (Size: 68.05 KB Mem: 509.5 MB)
? [Apr 09 13:56:18] Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 283605 bytes) in /var/www/vhosts/example.com/httpdocs/blog/wp-includes/wp-db.php on line 3122

Scan 3
? [Apr 09 14:13:35] Scanning contents: wp-content/uploads/123rf-52128799_s-450×300.jpg (Size: 21.33 KB Mem: 509 MB)
? [Apr 09 14:13:35] Scanning contents: wp-content/uploads/123rf-52128799_s.jpg (Size: 41.89 KB Mem: 510.25 MB)
? [Apr 09 14:13:35] Scanning contents: wp-content/uploads/123rf-52590276_s-150×150.jpg (Size: 4.35 KB Mem: 511.5 MB)
? [Apr 09 14:13:35] Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 1048577 bytes) in /var/www/vhosts/example.com/httpdocs/blog/wp-content/plugins/wordfence/lib/wordfenceScanner.php on line 303

Actually, it’s not entirely true that memory never got released. It got released just before starting scanning plugins (wp-content/plugins/) and uploads (wp-content/uploads/). I assume that if you release the memory after scanning plugins block and before starting uploads everything will work fine. Just a guess…

Hi @wfyann, I excluded all folders/files for the WP Rocket caching plugin and was able to complete the scan without errors. The maximum amount of allocated memory went to 410M. Not sure whether you want to add WP Rocket to the list of “conflicting” plugins as it doesn’t really conflicts, it just caches a lot of files locally. Proper solution would be to release the memory as it approaches the limit.