Crawler not caching pages?

  • Resolved colinfroggatt

    (@colinfroggatt)


    1 year, 9 months ago

    Hi. I am trying to get the Crawler to work on my site. It is successfully crawling all the pages but never seems to actually cache them.

    On the crawler summary I see all pages covered in blue (Successfully Crawled) but virtually non are green. In general, the green ones appear to be those manually visited.

    I can see the crawler visiting pages in the crawler log, but when I visit the page in a private browser as guest I see that all of the cache headers are misses for the page.

    x-litespeed-cache: miss
    x-litespeed-cache-control: public,max-age=172800
    x-litespeed-tag: 324_tag_priv,public:324_HTTP.200,public:324_book,public:324_URL./book/the-shamanic-journey/,public:324_Po.20737,public:324_
    x-powered-by: PHP/8.0.28
    x-qc-cache: miss

    Watching the crawler status only ever shows “…updated position” status e.g.:

    01 Mar 2023 13:18:03 Size: 115 Crawler: #1 Position: 22 Threads: 1 Status: crawling, updated position

    Any help or guidance here would be appreciated.

    LS Cache 5.3.3.

    WP 6.1.1

    Using LS Web Server (on greengeeks)

    thanks, Colin

    The page I need help with: [log in to see the link]

Viewing 5 replies - 1 through 5 (of 5 total)
  • Thread Starter colinfroggatt

    (@colinfroggatt)

    example record from crawl log (anonymised): 

    03/01/23 13:24:41.421 […:…. 1 ZWX] ?? ------GET HTTP/1.1 (HTTPS) /gloss/recap/
    03/01/23 13:24:41.421 […:…. 1 ZWX] Query String:
    03/01/23 13:24:41.421 […:…. 1 ZWX] HTTP_REFERER: https://www…co.uk/wp-cron.php?doing_wp_cron=1677676857.9830830097198486328125
    03/01/23 13:24:41.421 […:…. 1 ZWX] LSCACHE_VARY_VALUE: +ismobile+webp
    03/01/23 13:24:43.217 […:…. 1 ZWX] [Router] get_role:
    03/01/23 13:24:43.219 […:…. 1 ZWX] ?? Suppress default WP lazyload
    03/01/23 13:24:43.223 […:…. 1 ZWX] [Optm] init
    03/01/23 13:24:43.607 […:…. 1 ZWX] ?? [Tag] Add --- HTTP.200
    03/01/23 13:24:43.608 […:…. 1 ZWX] [Ctrl] X Cache_control init on
    03/01/23 13:24:46.846 […:…. 1 ZWX] [Core] Footer hook called
    03/01/23 13:24:47.388 […:…. 1 ZWX] [Core] Footer check passed
    03/01/23 13:24:47.388 […:…. 1 ZWX] [Router] get_role:
    03/01/23 13:24:47.389 […:…. 1 ZWX] [Router] get_role:
    03/01/23 13:24:47.389 […:…. 1 ZWX] [Vary] role id: failed, guest
    03/01/23 13:24:47.389 […:…. 1 ZWX] [Router] get_role:
    03/01/23 13:24:47.389 […:…. 1 ZWX] [Router] get_role:
    03/01/23 13:24:47.390 […:…. 1 ZWX] ?? X-LiteSpeed-Cache-Control: public,max-age=172800
    03/01/23 13:24:47.390 […:…. 1 ZWX] ?? 107.6.184.242:58684X-LiteSpeed-Tag: 324_HTTP.200,324_glossary,324_URL./gloss/recap/,324_Po.13586,324_
    03/01/23 13:24:47.404 […:…. 1 ZWX] [Core] Footer check passed
    03/01/23 13:24:47.404 […:…. 1 ZWX] ?? finalize
    03/01/23 13:24:47.420 […:…. 1 ZWX] [Optm] start
    03/01/23 13:24:52.157 […:…. 1 ZWX] [Router] get_role:
    03/01/23 13:24:52.157 […:…. 1 ZWX] [Vary] role id: failed, guest
    03/01/23 13:24:52.158 […:…. 1 ZWX] [Router] get_role:
    03/01/23 13:24:52.158 […:…. 1 ZWX] [Router] get_role:
    03/01/23 13:24:52.159 […:…. 1 ZWX] ?? [Tag] Add --- MIN.0a5d8c3cd4a7fd22bb1e6f82c83a0b79.css
    03/01/23 13:24:52.167 […:…. 1 ZWX] [Optm] js defer exclude https://www…co.uk/wp-includes/js/jquery/jquery.min.js?ver=3.6.1
    03/01/23 13:24:53.584 […:…. 1 ZWX] [Router] get_role:
    03/01/23 13:24:53.584 […:…. 1 ZWX] [Vary] role id: failed, guest
    03/01/23 13:24:53.584 […:…. 1 ZWX] [Router] get_role:
    03/01/23 13:24:53.584 […:…. 1 ZWX] [Router] get_role:
    03/01/23 13:24:53.585 […:…. 1 ZWX] ?? [Tag] Add --- MIN.2e3a91ce3cb042fb5a079786d56f3cc7.js
    03/01/23 13:24:53.610 […:…. 1 ZWX] Response headers --- array (
    0 => 'X-Powered-By: PHP/8.0.28',
    1 => 'X-DNS-Prefetch-Control: on',
    2 => 'Content-Type: text/html; charset=UTF-8',
    3 => 'Link: https://www…co.uk/wp-json/; rel="https://api.w.org/"',
    4 => 'Link: https://www…co.uk/wp-json/wp/v2/glossary/13586; rel="alternate"; type="application/json"',
    5 => 'Link: https://www…co.uk/?p=13586; rel=shortlink',
    6 => 'X-LiteSpeed-Cache-Control: public,max-age=172800',
    7 => 'X-LiteSpeed-Tag: 324_HTTP.200,324_glossary,324_URL./gloss/recap/,324_Po.13586,324_,324_MIN.0a5d8c3cd4a7fd22bb1e6f82c83a0b79.css,324_MIN.2e3a91ce3cb042fb5a079786d56f3cc7.js',
    )

03/01/23 13:24:53.610 […:…. 1 ZWX] End response
    Plugin Support qtwrk

    (@qtwrk)

    please provide the report number

    you can get it in toolbox -> report -> click “send to LiteSpeed”

    Thread Starter colinfroggatt

    (@colinfroggatt)

    Report number: AOOJQGGH

    Report date: 03/02/2023 11:31:56

    thanks. C

    Plugin Support qtwrk

    (@qtwrk)

    could you please also do me a full screenshot on your crawler summary page ?

    Thread Starter colinfroggatt

    (@colinfroggatt)

    Hi @qtwrk , thanks for the reply and apologies for taking so long to respond. We just upgraded hosting. An updated report number blow for the new host.

    Report number: VIGWOFVJ

    Report date: 03/08/2023 15:34:43

    Also updated screen shot on new host of crawling page here: https://snipboard.io/e3Bf0X.jpg

    thanks, Colin

Viewing 5 replies - 1 through 5 (of 5 total)
  • The topic ‘Crawler not caching pages?’ is closed to new replies.