Crawler schedule via Cron job + pagination caching

please provide the report number , you can get it in toolbox -> report -> click “send to LiteSpeed”

for pagination , not sure how your theme works , but I have seen some theme that will have individual URL , but loads as pagination , in such case , if you can add these page’d URL into sitemap , then it also works.

Hello,

I am using Woodmart Theme.

Report number is BUCABJQB.

I actually found a way for scheduling the crawl to cache the site : by using a PHP script to empty the cache at a certain period (since Litespeed cache expires once a week, I scheduled a cron job each week). Once the cache is emptied, litespeed crawl automatically starts, which is what I want.

The pagination system on my website is standard (no “load more” button, or infinite scroll), here is a screenshot: https://prnt.sc/Z1W2NsJDkynP

Pagination URL structure is like this : /page/**page_number ** (for example : /page/1, /page/2, and so on). I can’t add pagination to the sitemap, as it is against SEO recommended practice. Is there another way to preload pagination?

For example, the WP Rocket plugin can be configured to preload pagination : once a user click on a page number, WP Rocket will add this page to the database, and next time that preload will occur, the page number will be included in the preload. You can find more details here (“URLs automatically excluded from the Preload” section) : https://docs.wp-rocket.me/article/8-preload-cache

Here is the code used by WP Rocket :

function enable_preload_for_pagination( $regexes ){
		global $wp_rewrite;
		$pagination_regex = "/$wp_rewrite->pagination_base/\d+";
		$exluded_urls = [		
			$pagination_regex,
		];
		$regexes = array_diff( $regexes, $exluded_urls);
		return $regexes;
	}
add_filter( 'rocket_preload_exclude_urls', 'enable_preload_for_pagination');

Contrary to WP Rocket, Litespeed does not save the pages number in database, so a new preload will no do any pagination caching. Cache will only occur if a user clicks on a page number.

How to solve this ?

• This reply was modified 10 months, 1 week ago by daivymerlijs.
• This reply was modified 10 months, 1 week ago by daivymerlijs.

I don’t have solution for that , currently LSCWP relies on 3rd party sitemap ??

Then it would be great, at least, to save the page numbers in the db once a user click on it for the next preload to include it.

This is really surprising that a lot of cache plug-ins don’t do that. What about big e commerce websites with a lot of page numbers ?

• This reply was modified 10 months, 1 week ago by daivymerlijs.

a ha , good question , I wonder how does sitemap plugin handles it ?

Rankmath and all SEO plugins does not include pagination in sitemap, because it is against Google’s recommendations.

It would be great for Litespeed to have an option for preloading sitemap, in a way similar to WP Rocket as I detailled above.

yeah, now that is even more weird , if it’s not good for SEO , then why theme does that , let alone the sitemap generators don’t do that either

I will forward the case to our devs for their assessment and evaluation.

woodmart theme does not include any pagination preload, it uses AJAX.