• Resolved daivymerlijs

    (@daivymerlijs)


    Hello team,

    First, congratulations for your plugin that seems to solve a lot of limitations of many of the crawlers embedded in famous cache plugins.

    However, I have two suggestions that can be used to improve the plugin :

    1) Sitemap priority : this option would force the crawler to crawl the pages included in sitemap first. Why ? Because thoses pages are the “top priority” for crawling : caching pagination and query is fine, but is not the “priority” while crawling. So, children pages should be crawled after sitemap’s pages have been crawled.

    2) Crawler scheduling : instead of period interval, we should be able to program the crawler to start at specific times (for example, every monday at midnight). Use case : caching the full website during the night, while the numbers of users is limited.

Viewing 10 replies - 16 through 25 (of 25 total)
  • Thread Starter daivymerlijs

    (@daivymerlijs)

    @tmmtechnology

    Yes, it works well now and the sitemap priority setting is a nice addition.

    The only feature I can’t wait to use is the WP-CLI + cron.

    A PHP script with a cron job will be a good too, instead of WP-CLI (if too difficult to implement for now). For example, WP Rocket allow users to purge the cache with a cron via a PHP script : https://docs.wp-rocket.me/article/494-how-to-clear-cache-via-cron-job

    Maybe you could try something similar.

    Thread Starter daivymerlijs

    (@daivymerlijs)

    @tmmtechnology

    I think I noticed a problem for the “Cookies” part.

    For your crawl to act like the Litespeed crawl with guest mode enabled, I need to put the following cookie :

    _lscache_vary=guest_mode%3A1 (The “%3A” is the encodedURL version of “:“.)

    However, it is not possible to paste the %3A part (the textbox remove it when saving) and it seems that your plugin does not have a function that can encode the “:” ; because when I use the “_lscache_vary=guest_mode:1″, your plugin does not seem to use the encodeURL function, so the cookie can’t work.

    Can you please fix that ?


    Thread Starter daivymerlijs

    (@daivymerlijs)

    Another plugin suggestion :

    it would be great that the crawler does not crawl again pages that he already visited once during his warming session. I clearly don’t see the point to crawl the same pages several times during the same session.

    For now, there is the “Skip pages with warmed canonical URLs” but it will be even better to have a “Skip already warmed pages” ; as I am concerned, I use filters to displaying my products and I would like those pages to be crawled (but they can’t since their “canonical url” is already warmed)

    Plugin Author TMM Technology

    (@tmmtechnology)

    Thanks. We will check and respond to all your messages a bit later.

    Thread Starter daivymerlijs

    (@daivymerlijs)

    Another plugin suggestion :

    “Most wanted urls” function, similar to kitt crawler. Basically, the crawler should crawl the pages that are most visited by users, and should crawl them first.

    Hi @daivymerlijs,

    I’m trying to fix a similar issue – _lscache_vary=guest_mode%3A1 (The “%3A” is the encodedURL version of “:“.) Were you able to resolve it (or still waiting)?

    I’m trying to also find the correct settings that will work with LSC guest mode but unsuccessful so far – still getting x-litespeed-cache: miss. Any input appreciated?

    Plugin Author TMM Technology

    (@tmmtechnology)

    @avenja I don’t see that you made a support topic.

    Please make one I will pay more attention to your request. Though I can’t promise to find a solution as didn’t have much experience with LSC, but maybe someone else will respond.

    1 issue = 1 topic.

    Plugin Author TMM Technology

    (@tmmtechnology)

    @avenja won’t using “:” work?

    I don’t have LSC installed, but from googling quickly I think something like this could work?

    Plugin Author TMM Technology

    (@tmmtechnology)

    Plugin Author TMM Technology

    (@tmmtechnology)

    @daivymerlijs

    I added a new logic. If you want to start with WP CLI, you can use wp eval and this code:

    • Start:?\Cache_Warmer\AJAX::start_warm_up( false );
    • Stop:?\Cache_Warmer\AJAX::stop_warm_up( false );
Viewing 10 replies - 16 through 25 (of 25 total)
  • You must be logged in to reply to this topic.