LiteSpeed crawler CronJob (not working)

  • Resolved gopa4

    (@gopa4)


    10 months, 2 weeks ago

    Hello,

    I am on a LiteSpeed server but they did not have this function enabled which is needed for the crawler to pre-cache the pages.

    My crawler in the LiteSpeed is enabled. If I click on Manual, nothing happens…

    https://ibb.co/9TKPVnF

    I have tried this recommendation:

    Here, you can set up a cron job by inputting a ‘crontab’ command. The cron job command to execute might be site-specific, LiteSpeed’s crawler uses this URL format: https://<your_site>/wp-admin/admin-ajax.php?action=litespeed-crawler-cron

    So the command should be something like:
    */20 * * * * wget -q -O - "https://<your_site>/wp-admin/admin-ajax.php?action=litespeed-crawler-cron" > /dev/null 2>&1

    but it did not work. I got an error. When I visit the link in the browser, I see the output 0.

    Any suggestions what I can do to solve it?

    Thanks in advance!

Viewing 3 replies - 16 through 18 (of 18 total)

  • Correct the delay setting as indicated. Ask you provider how many cores the CPU of your server have and devide the number by 2 and set this value in Server Load Limit.

    Thread Starter gopa4

    (@gopa4)

    ?I see this: NOTE: Server allowed max value:?5.2

    I changed to 3 and pushed Manually

    Current sitemap crawl started at: 54m 10s ago

    The next complete sitemap crawl will start at: 05/02/2024 11:44:32

    Current crawler started at: 54m 10s ago

    Current server load: 6.58

    Last interval: 54m 10s ago

    Ended reason:?end

    I see that “The next complete sitemap crawl will start at:?” will start in the past (not in the future)..?

    Thread Starter gopa4

    (@gopa4)

    Thank you for your explanation!

    The crawler started to work (slowly…) Screenshot> https://ibb.co/5TQVzzs

    but I the most pages are not pre-cached.. Crawler Status 200.

    Should I switch the crons off (or not) to make it faster?

    My cache is purged every night. As I understand, I need to know how long it takes to crawl the website so I can put it in “Crawl Interval”? Can you please check these settings https://ibb.co/X7yg9pQ

    According the map is this post should be pre-cached https://ibb.co/TMM1yr7

    but it does not https://ibb.co/7j4NWKc

Viewing 3 replies - 16 through 18 (of 18 total)
  • The topic ‘LiteSpeed crawler CronJob (not working)’ is closed to new replies.