Preloading subpages, sitemap index (Yoast), overview of preloaded pages

Hello @dajanas

Thank you for your inquiry and I am happy to assist you with this.
What you need to make sure is that your sitemap contains all URl’s
You have to supply the main sitemap, which could be an index too. The plugin will look for <sitemap>, which specifies another sitemap URL, and parse that URL as a sitemap too. So “nested” sitemaps are no problem.
If you are using native wp-cron possibly some pages are bein missed do the solution for this is to disable wp-cron by adding define('DISABLE_WP_CRON', true); to wp-config.php and Once this is done, create a cron job on your server that will call wp-cron.php at a set interval.
Or better yet you can use wp-cli so, instead of calling wp cron via crontab – you should call it directly via
wp w3-total-cache pgcache_prime
and track the output which pages/sitemap positions are primed by script
I hope this helps!
Thank yo!

Hello,

thanks for your answer. I already disabled WP-Cron and I am using Cronjob on my server (which works).

But on the Internet, so many websites say that W3 Total Cache can’t handle index sitemaps – like Yoast’s sitemap (“sitemap_index.xml”). Therefore I am confused.

But point 1 of my question is also very important to me, I’ll gladly repeat it again:

“I noticed that although the pages of the sitemap are “preloaded” and thus cached, the subpages are unfortunately not. Example: The first page of a category is automatically cached by cron – but the subpages 2, 3, 4, 5 etc. unfortunately not. How can I set it so that the subpages are also cached when “preloading”? Because at the moment the first page of a category loads quickly (because it was cached), but pages 2, 3, 4, 5 etc. are not. The same applies to subpages of tags.”

And point 3 is also very important to me:

“Is there a way to see which pages have been “preloaded” by W3 Total Cache? Some kind of overview or coverage of already cached pages without visiting the respective website-page and without looking into the source code?”

Regards

Hello @dajanas

I don’t know which websites you are referring to but you can use Yoast sitemap. In fact, it’s recommended to use it.
You need to make sure that the sitemap contains all the pages including those sub-pages. If those sub-pages are not in the sitemap thy will not be preloaded.
And as for point 3, as I mentioned in the previous post, the only way is to use wp-cli
You should call it directly via
wp w3-total-cache pgcache_prime
and track the output which pages/sitemap positions are primed by script
Thank you!

This is a pity, because Yoast does not create subpages in the sitemap. Which makes sense, too, because subpages should not be indexed by Google, because otherwise it is possible to have duplicate content. But for Cache it is important (and should be independent of pages to be indexed). The cache should (in my opinion) automatically cache the subpages as well. Maybe a new feature?

Hello @dajanas

Thank you for your reply.
This is simply how W3 Total Cache works. Cache Preload needs to be feeder with the sitemap or else W3 Total Cache would not know which pages to preload and cache.
You can supply the main sitemap with additional pages. The plugin will look for <sitemap>, which specifies another sitemap URL and parse that URL as a sitemap too. So “nested” sitemaps are no problem.
I understand the issue with indexed pages but this is how Cache Preload works.
I’ll speak to the team and check if we can do something about this but at the moment this is the only way.
Once again Thank you for your questions and your feedback.

Okay, I understand. But under “Purge Policy: Page Cache” you can find the option “Purge limit” with the description: “Specify number of pages that lists posts (archive etc) that should be purged on post updates etc, i.e example.com/ … example.com/page/5.
0 means all pages that lists posts are purged, i.e example.com/page/2…”

So obviously ABCD is able to detect and (with this option) automatically purge subpages. So why should it not be possible to define that subpages of archives, tag pages, categories are cached automatically as well?

Do you know what I mean?

Hello @dajanas

Yes, I understand what you are saying but those are two completely different things and work differently.
Cache Preload requires the sitemap to specify those pages and purge policy doesn’t.
This is simply the case of how the Cache preload works.
Thanks!

Okay, do you know a plugin that is able to create a sitemap, including ALL subpages (page/2, page/3, page4… etc.)? I would use this specific sitemap just for W3 Total Cache then.

Hello @dajanas

Sorry for the late reply.
I’ve personally never done this before but I believe Yoast has an option for that:
https://yoast.com/help/how-to-customize-the-sitemap-index/
Thanks!