Expires and Cache-Control headers overwritten or not sent

Hello @dstokes

Thank you for yoru questions and I am happy to answer.
What you are trying to do cannot be achieved with W3 Total Cache. Simply because you cannot specify the pages or the URLs that can be excluded from the Browser Cache rules,
in your case, cache control, expires header, and Last modified.
So the only way for this to work is to completely disable W3 Total Cache BC, or simply adjust the settings in W3TC to match the settings needed for expiry.
The reason for this is that the Browser Cache rules are set in the .htaccess file and using more than one set of rules for the same header can cause problems.
Let me know if this helps!
Thanks!

Yes, that helps. I looked at my .htaccess file and I can see that W3TC modifies it when “Set cache control header” is turned ON. So it makes sense that if I want to assign any headers programmatically then I need to turn that setting OFF.

But I’m left with a question: How can I signal W3TC that a cached page should be considered stale at a certain point in time? I had hoped that using either an expires header, or max-age inside a cache-control header would cause W3TC to discard the cached page after that point. I know that those headers pertain to the browser cache but I was hoping that W3TC would look at and respect those settings for the page. But that does not seem to be the case. I feel like I must be missing something. Is there some other way that I can tell W3TC to automatically purge a cached page after a specific time?

What I’ve done is set up a scheduled event inside WordPress to purge the cached page at the right time. But the scheduled event will only run when WordPress receives an external request, and that doesn’t happen because W3TC is intercepting those requests and returning the cached pages! What I need is for W3TC to recognize that the requested page becomes stale at a specific time and automatically purge the page. How can I do that?

Also, can you tell me why W3TC does not resend those headers (expires and cache-control) when it sends a cached copy of the page? Is it because W3TC relies entirely on .htaccess to set those headers and never adds headers programmatically? But wouldn’t it be considered a W3TC bug if the cached page is not an exact copy of the original in all respects including its headers? It would be possible for W3TC to add those headers programmatically, whether .htaccess does or doesn’t overwrite them. Either way the result would be identical to the original page. I believe this may also solve my problem because if the browser has the expires information then a subsequent browser request should inform the cache that the page has expired. Does that make sense or am I missing something?

Thanks for your help!

Hello @dstokes

Thank you for your questions.
When using Disk: Enhanced, the page is cached until invalidated – meaning until the cache is purged for some reason, manually or according to the Perofrmance>Page Cache>Purge Policy.
WIth Memory based caching you can set the objects to expire in a period of time.
You can purge the specific URL manually using on a specific time using the server cron and calling w3tc_flush_url($url); or using wp-cli:

/**
	 * Clear something from the cache.
	 *
	 * ## OPTIONS
	 * <cache>
	 * : Cache to flush
	 * all         flush all caches
	 * posts       flush posts (pagecache and further)
	 * post        flush the page cache
	 * database    flush the database cache
	 * object      flush the object cache
	 * minify      flush the minify cache
	 *
	 * [--post_id=<id>]
	 * : flush a specific post ID
	 *
	 * [--permalink=<post-permalink>]
	 * : flush a specific permalink
	 *
	 * ## EXAMPLES
	 *     # Flush all
	 *     $ wp w3-total-cache flush all
	 *
	 *     # Flush pagecache and reverse proxies
	 *     $ wp w3-total-cache flush posts
	 */

As for the Browser cache, Whatever cache headers are set in a browser are used by the client until they expire or something changes that invalidates them, such as a hard refresh.
You can try https://httpd.apache.org/docs/2.4/mod/core.html#files <Files>, <FilesMatch>, <Directory>, or <DirectoryMatch> to set headers for specific pages.
I hope this helps!

Thanks for your help with this. I understand it much better but I’m still having a problem.

I am using disk caching.

I tried to purge the cache using server cron and calling w3tc_flush_url($url). But that does not always work because the WordPress cron implementation only executes when a request comes into WordPress, and there may be no requests except those intercepted by the cache. The operating system does not run the job, WordPress does. But W3TC may intercept all incoming requests and return cached pages, in which case WordPress does not run and neither does the cron job.

I need the site to operate properly all the time, even when there is no admin logged in, nobody editing posts etc. or otherwise accessing WordPress and causing the cron job to run or the cache to be purged based on policies. The only activity may be non-logged in users accessing cached pages. But the cached event page still must expire when the event begins. Policies don’t work and cron doesn’t work in the general case.

The only solution I can think of is to add a link to refresh the event page with a url that includes a query string like “?refresh”. That will bypass the cache and give WordPress a chance to run the cron job and purge the cache. It’s not a great solution but I think it works based on my testing so far.

It would be nice if W3TC had a way to establish an Expires date/time for a cached page. It seems to me there could be a setting and then simply use the Expires header if one exists. Or alternatively, a setting to allow incoming requests to always trigger the WordPress cron system to execute. The latter is more complicated and would slow down the system slightly so the former would be better.

Is either possible?

Thanks again!