Preload on a high traffic site

100 or 100,000?

Preload is configured to refresh cache with 720 minutes. Is it the recommended?

If the server doesn’t feel the pain of the preload, then yes.

Hi, @cuta. 100k per day.

On each 30 minutes (from 8:00 AM until 11:00 PM) we have a new post. So, should I preload all those new posts when it’s published to have the benefit from preload? This is my doubt.

As we have a lots of posts during a day, using default configuration, the last posts won’t be preloaded when they are still new.

About my dedicated server:

4 CPU / 8GB RAM

Caching is done automatically upon enabling the preload mode from the newest post to the oldest post as I am aware, but it takes some time to cache the contents. I would more worry about the oldest contents than the newest contents.

Got it. Your setup is bit complex but not unusual.

should I preload all those new posts when it’s published to have the benefit from preload?

Yes.

Preload is configured to refresh cache with 720 minutes.

Please reduce this to the lowest possible value. If WP Super Cache doesn’t allow it, then never mind.

If you have currently enabled to “preload all posts”, then you may consider reducing it to fit the top 50 posts (or whatever the daily number), so that the latest posts are always cached.

Also, the cache-control header from WP Super Cache can be modified to improve the hit rate in CloudFlare. Currently, it is “Cache-Control: max-age=3, must-revalidate” that could modified to “Cache-Control: max-age=300” (for example). This way CloudFlare can cache an article as much as 5 minutes. You may tweak it to fit what works for your particular site.

From what you have mentioned, I think that something else is causing the trouble for “not” having the cached version of latest posts. You might want to check all other settings. It is certainly not “preload”.

I hope that helps.

Thanks for support.

I changed cache control in ../wp-content/cache/.htaccess. Is it right?

It’s my pleasure to share what I learned.

“must-revalidate” header is still present. It means Cloudflare needs to re-validate its cache for every request irrespective of the max-age value. You may remove “must-revalidate” header, as long as you use Cloudflare (or any other similar layer). Once removed, please see the hit rate in Cloudflare. And then, you may consider improving the max-age further.

Please know that it is still important to have the “must-revalidate” header, if you remove Cloudflare in the future. Assume that you don’t use Cloudflare and you don’t have “must-revalidate” header. Now, there is no way to let the browsers / visitors know that there is an updated version of an article (mostly home page), until the “max-age” value expires.

Basically, if you can understand what each of those headers do, you’ll be in a better position to fine-tune your configuration to fit your particular site.

Also, please know that I haven’t mentioned about purging the Cloudflare cache. I hope you have a plugin or something to purge the cache. Without such a system in place, using max-age to a very high value (such as 24 hours) may backfire by sending the stale / old data to the visitors.

hmmm! I understood, Pothi.

I use cloudflare and removed now “must-revalidate”. I set max-age to 600 (10 minutes). Do you think is a good time for my scenario?

Thanks again! =)

I use cloudflare and removed now “must-revalidate”. I set max-age to 600 (10 minutes). Do you think is a good time for my scenario?

Yes. There are always better ways to fine-tune an installation. For example, Cloudflare doesn’t cache dynamic content by default. Ref: https://support.cloudflare.com/hc/en-us/articles/202775670-How-Do-I-Tell-CloudFlare-What-to-Cache- . Assuming that you have already setup page rules with Cloudflare, you may go a step-further and setup only “s-maxage” header (and leaving max-age=0) so that your visitors’ browsers will never cache content even for those 10 minutes and will always look for fresh content (from Cloudflare). If this looks too complex, please leave the current setup as it is. The current setup looks much better already.