Editing a page: Is there an option to not only purge but regenerate its cache?

WordPress → Performance → Page Cache → Cache Preload:

? Automatically prime the page cache — I left it OFF
Update interval: 900secs
Pages per interval: 10
Sitemap?URL: empty

?? Preload the post cache upon publish events — Set it from default OFF to ON.

Thoughts

“Automatically prime the page cache” seems to my like a job which caches every URL in the provided Sitemap. (“Polling”)

“Preload the post cache upon publish events” is obviously event-bound (“Pushing”). And editing is a “publishing event”. So I expected this to maybe do what I was looking for. But it didn’t.

Observing cache directory/files in FTP client while activities in web browser:

• Editing and saving page in WordPress Block Editor.
• The purging works: Cached html and gzip files get “_” prefix and “_old” suffix and a modification date of 2016-11-23 (no date that seemingly has anything to do with my page, maybe a constant from W3TC ?)
• But no new cached html/gzip pair gets regenerated.
• Only when the URL gets requested by a new visitor the files again get a normal filename and current moddate.
• And every subsequent visitor/requests gets it as a HTTP 304.

Hello @abitofmind

Thank you for reaching out and I am happy to help!

The only way to do this is to keep Cache Preload working. There is no option or a hook to preload a specific page once the cache for that page is purged, so the only solution is to enable Cache Preload. ?Preload the post cache upon publish events only works if the Preload is enabled which in your case is not. In order for the Preload to work you also need to add the sitemap URL in the SItemap URL field.

What preload cached does is:

?Check the last offset
– Check how many URLs are allowed to be processed per run
– Fetch all URLs from sitemap (even nested sitemaps work)
– Loop through a set of URLs to visit them

If the end of the list is reached, it will start from the beginning in the next run.

You can also use wp-cli to for preloading. so, instead of calling wp-cron via crontab – you should call it directly via
wp w3-total-cache pgcache_primeand track the output of which pages/sitemap positions are primed by the script

Calling the wp-cli command is the same as the cronjob, but if you want you can specify where to start this run and how many pages should be visited (instead of using the configuration values)

The HTTP?304 Not Modified?client redirection response code indicates that there is no need to retransmit the requested resources. It is an implicit redirection to a cached resource. If this is enabled in the W3 Total Cache it’s added as a header if the content is not changed and the page is cached.

I hope this helps!

Thanks!

1) Thanks for explaining how the mechanism works!

Proposal: With better layout in W3TC Settings → Page Cache → Cache Preload the individual settings and their interdependency could be intuitively clear to the user:

2) I think the current behavior could be improved in terms of efficiency and also for ideal UX for editorial use cases.

• a) Preload cache is a job which goes through all URLs in the sitemap and regenerates them. It’s a pull/polling approach (by nature inefficient).
• b) If a user edits a page/post (an event) we know with 100% certainty that the corresponding cache should be rebuild. It’s a push / event-driven approach (by nature most efficient).
  
  Could you think of changing your architecture that b) not necessarily requires the active job to be necessary? So have one master toggle “Preload Cache” and then two sub options “By Job” and “By publishing events”.
  
  My use case: I’m editing a lot of pages. As an editor I want to be able to benefit from the cached speedup in the frontend too without that first slow response. Cause even though the Block Editor is closer to WYSIWYG, still I check most pages out in the frontend too. And also I click around from there to see how that new/changed page fits into the whole website.
  
  Then needing to wait for 1200ms server latency each time as opposed to only 20ms for the cached version makes a huge accumulated difference when per day performing ca. 200 edits and subsequent 500 mini browsing sessions.
  
  What do you say?

• This reply was modified 1 year, 6 months ago by abitofmind.
• This reply was modified 1 year, 6 months ago by abitofmind.

Preload the post cache upon publish events only works if the Preload is enabled which in your case is not.

I’m not sure I yet understand it entirely. If I have both “Automatically prime the page cache” (the “master toggle” for Cache Preload) and “Preload the post cache upon publish events” active, my understanding is that:

1. If the job kicks in (by interval or manual wp-cli invocation) it runs through all URLs on the sitemap (with the offset and batch-size limitations). So possible split onto several runs.
2. Does this regenerate each page fully? (efficiency concern in general and for the next question in particular)
3. If at a later point I edit page-X what happens?
   a) Only page-X is regenerated.
   b) The whole job gets started (page X will also get picked up somewhere in that job).

3b, if indeed true, seems like a huge waste of computing resources to me. I hope 3a is what happens. Then simply having both options on would achieve the behavior I need as a frequent editor/reader.

Hello @abitofmind

Thank you for your suggestions.

Again, once the post/page is updated, the cache for that page is purged, and the Preloader should begin from that page, and preload the rest of the website depending on the sitemap added.

The preload does not stop, and once the entire website is preloaded, the preload starts again from the top, or again, if the page is updated, it starts from that page.

Thanks!

After I overcame how to get my wp-sitemap.xml as standard compliant application/xml into the “Sitemap URL” field I was able to test your proposed setup in practise.

My settings meanwhile:

• ?? Automatically prime the page cache
• Update interval: 10 seconds (wanted quickly noticeable results, default 900)
• Pages per interval: 10 (default left as-is)
• Sitemap URL: https://example.com/wp-sitemap.xml
  Proposal: Support absolute path like /wp-sitemap.xml for hassle-free WordPress database migrations.
• ?? Preload the post cache upon publish events

Your recent answer clarified further aspects for me:

• But my observation contradict some of it (footnotes):

Again, once the post/page is updated, the cache for that page is purged, and the Preloader should begin from that page ⁴, and preload the rest of the website depending on the sitemap added.

The preload does not stop ², and once the entire website is preloaded, the preload starts again from the top ², or again, if the page is updated, it starts from that page ⁴.

Your info (claims have footnotes) and my experiences with it (corresponding to footnotes where applicable)

• 1) Does W3TC provide any tools to monitor “what it currently does” in regards to Preloading?
• My primitive monitoring was to observe folders/files within /wp-content/cache/page_enhanced/ and their moddate/filename changes.
• 2) As soon as I clicked “Save Settings & Purge Caches” the first Preload Cache loop started running seemingly (as observed from filesystem changes)
• a) … which every “Update Interval” (seconds) performing “Pages per interval”?
• b) Is this an infinite loop running 24/7?
  Your recent answer strongly indicates that.
• c) Or does this only happen as a oneshot when I click “Save Settings & Purge Caches” or when a publishing event happens (if “Preload the post cache upon publish events” is ON) ?

From my observations I think it is 2c

• That folder corresponding to page-slugs got the current moddate and its html/gzip file pair got filename suffixed with “_old” and their moddates changed to 2016-11-23.
• But other files/folders got recreated first.
• That edited page got recreated “in the usual queue order” of the processing job. Only then it got re-created.
• But after another user-event-triggered batch processing full run was completed no new file/folder moddate change occured anymore. So I think in total not an infinite loop.

My critique (with my limited knowhow)

• 4) If you edit a specific page/post it should not only be purged, but also be the first which gets preloaded (cached) again from the whole batch.
  a) You as an editor want particulalry the page you just edited immediately available cached in the FrontEnd to view after editing in the Block Editor.
  b) Your readers should first get the cache renewed where the latest change occured. Instead of the latest change just triggering a global page-cache-rebuilding which gets processed in the usual order which not at all corresponds to most recently changed content.
• 5) Why does the editing of a particular page trigger the preloading of ALL (!) URLs within the sitemap?
  Seems very undifferentiated/inefficient!
  a) Contrary to that at section “Purge Policy” you can very fine-grained control which other pages/feeds get purged when a page/post gets edited.
  b) Why not the exact fine grain control for preloading only certain aggregation feeds after a post edit? And maybe an even more relaxed after a page edit?
  
  As “Preloading” is now feels very stubborn, raw & inefficient to me.
  
  Update: Answer to myself: If you edit a page, it can have side-wide consequences: E.g. it’s site title, and that title is its menu entry, then you need to re-render every single blogpost or page of your website which has the menu! OMG, without Preload, in that example all other pages would still show the old page name in their menu…! Caching fine grained and reliable ain’t easy… Now I understand why you choose “the big hammer approach”. Inefficient but at least reliable. With prioritization in your sitemap you can at least influence which pages get regenerated first.

• This reply was modified 1 year, 6 months ago by abitofmind.
• This reply was modified 1 year, 6 months ago by abitofmind.
• This reply was modified 1 year, 6 months ago by abitofmind.
• This reply was modified 1 year, 6 months ago by abitofmind.
• This reply was modified 1 year, 6 months ago by abitofmind.
• This reply was modified 1 year, 6 months ago by abitofmind.

If in settings “Browser Cache” your “Cache Control Policy” for “HTML & XML” uses no “max-age” then the entire Preload processing loop only ever gets triggered from publishing events?

• This reply was modified 1 year, 6 months ago by abitofmind.

Hello M@abitofmind

Thank you for your feedback, and thank you for your research and the answer you provided for yourself and other potential inquiries about the preload.

Just to add, if you do not want to rely to wp-cron, the best is to use wp-cli as mentioned in the previous posts. With this approach you can have more freedom on what is preloaded and you can use the log to check what is prreloaded.

Browser Cache and Page Cache have some connections, however, the preload is not impacted by the Browser Cache. The preload process will loop once the end of the list is reached.

Thanks!

That this thread’s knowhow gain on Cache Preload does not remain isolated here, please also add it to your documentation! After eventually discovering your hidden gem of a documentation I looked there for “Preload”:

• On the docs overview it doesn’t appear at all.
• At section Page Cache (its topical parent) no mention.
• But in W3 Total Cache Page Caching FAQ there are a few words on Cache Preload, but not too many.

—

You wrote earlier:

What preload cached does is:

1. Check the last offset
2. Check how many URLs are allowed to be processed per run
3. Fetch all URLs from sitemap (even nested sitemaps work)
4. Loop through a set of URLs to visit them

You wrote just now:

Browser Cache and Page Cache have some connections, however, the preload is not impacted by the Browser Cache.

Thanks for confirming explicitly. I was not sure whether by “offset” you meant age as in “time offset”. Now I’m certain offset means the ordinal number in the URL-List, where it left off.

Fine that this is confirmed. I re-cap: In Browser Cache the settings Caching Policy & Expires header lifetime (aka Max-Age aka Time-to-Live aka TTL) have no influence on when / how often the Preload Loop loads.
Note: Even though it would seem plausible to sync those values meaningfully, there is certainly potential there. For now it remains an expert’s responsibility that those timing values line up well so that no/minimal edge cases result of it.

The preload process will loop once the end of the list is reached.

Ok not an infinite loop. Fine! My overnight observation confirm this.

Remaining questions for the Cache Preload documentation:

Please list all events that can kick off the Cache Preload. To my understanding:

1) Page Cache → Cache Preload → “Preload the post cache upon publish events” is ON. You create/edit/delete a page/post → Triggers a Preload Cycle.

2) Page Cache → Advanced → Garbage Collection Interval
Interval kicks in (default: Every 3600 secs (1 hour)) → Triggers a Preload Cycle.

Btw, please improve the field’s inline help somehow like this:

Beware of the interplay with the Expires header lifetimes (aka Max-Age aka TTL) which you can set for the different file types (HTML/XML, CSS/JS, Media&Other) at Browser Cache.

E.g. If you set the HTML files Max-Age higher than the garbage collection, then your garbage collection will renew them in time, but the clients will never get the renewed cached page b/c in their local cache it is still fresh (=below Max-Age). → lose control of a URL

And/or state other problematic timing value conflicts and traps.

Hey @abitofmind

Thank you for your feedback.
We’ll make sure to make more in-depth documentation about this!

Thank you again!

Please list all events that can kick off the Cache Preload.

• I stated my understanding of it. Which may be wrong or incomplete.
• You did not comment on that. Please. Would be appreciated.

Changed the thread back to “Not resolved”. Would appreciate an answer to my last comment. Thanks!

Hello @abitofmind

Once again I am sorry for the late reply. I’ve missed this for some reason.

To answer your questions:

1. Page Cache → Cache Preload → “Preload the post cache upon publish events” is ON.?You create/edit/delete a page/post →?Triggers a Preload Cycle. – Yes that is correct, however, as I’ve mentioned before, wp-cron is not very reliable, so it depends on the time between if the preload is currently taking place, and when the new content is published.
2. Page Cache → Advanced → Garbage Collection Interval
   Interval kicks in (default: Every 3600 secs (1 hour))?→?Triggers a Preload Cycle. – NO, Garbage Collection Interval only clears the _old files if they exist.
3. What triggers a preload is a the first visti to the website if the wp-cron is used, custom cron on the server if you ar enot using wp-cron or the wp-cli wp w3-total-cache pgcache_prime

Thanks!

Thanks for confirming. So regarding Preloading W3TC has indeed UX improvement potential:

Regenerating edited page immediately with top priority would benefit UX for the first requestor of the updated page

• Filed this issue on Github on 2023-05-08 but got no reaction so far.
• Please discuss this low hanging optimization fruit in your company. Thanks!

• This reply was modified 1 year, 5 months ago by abitofmind.