LiteSpeed Crawler and Pagespeed Issue

  • Resolved aaronwp03

    (@aaronwp03)


    1 year, 8 months ago

    Hi,

    Report number: PAMAXKBA

    Report date: 03/30/2023 11:46:41

    Crawler not working and more details on the report.

    Pagespeed error when litespeed is activated.

    How can this be resolved? Thank you.

Viewing 15 replies - 1 through 15 (of 18 total)

  • Which error is displayed at PageSpeed and why it is crawler related?

    Thread Starter aaronwp03

    (@aaronwp03)

    I am not tech-inclined but my site when being tested on pagespeed displays all error. I am thinking maybe there is something wrong in whatever configurations that is why the crawler is not working properly as well. Maybe they are connected? But my main concern is the crawler is not working, don’t care about the pagespeed results for now. Just mentioning whatever I can maybe it could help the support.

    https://prnt.sc/I6eKrFGFQqoL

    Could you offer a link to your page?

    Thread Starter aaronwp03

    (@aaronwp03)

    Hi,

    This is my homepage: https://cashflowproject.ca/

    First of all cache works at your page, but for whatever reason PageSpeed has any problems to connect your site. This also happends if PageSpeed us used in Chrome browser. (Lighthouse tab in dev console). Are you using any custom feature, rewrite rule in .htaccess or plugin that prevents/blocks access from bots or test sites? For example a security plugin?

    For me, this is not a cache plugin issue.

    Thread Starter aaronwp03

    (@aaronwp03)

    I don’t have security plugins like that, just the normal login using BuddyBoss.

    How about the crawler? I do have this: https://prnt.sc/jWE-T3XCZPhA

    I assume you tried to start the crawler manually? If yes, press the button for manual run twice.

    Unfortunately, I can’t help you with the other problem with PageSpeed. In any case, it is not a problem of the cache plugin.

    Thread Starter aaronwp03

    (@aaronwp03)

    Yes I did. I also tried to press the button for manual run twice still the same.

    Unfortunately, I can’t help you with the other problem with PageSpeed. In any case, it is not a problem of the cache plugin. Thank you.

    1. Did you enter the complete URL to the sitemap in crawler configuration?
    2. Do you see any error hints in crawler summary that crawler isn’t allowed at your host?

    Thread Starter aaronwp03

    (@aaronwp03)

    I created my sitemap from https://www.xml-sitemaps.com/

    downloaded the file and uploaded to my public_html folder. That is the url of my sitemap settings under crawler (https://hostinger/72561e6a5bfb1bdg/files/public_html/sitemap.xml).

    I am getting No valid sitemap parsed for crawler.

    Also, my server is https://www.hostinger.com/cloud-hosting. I think it’s designed for Litespeed and I asked the support it’s crawler supported.

    The given URL to the sitemap.xml contains only 2 URLs. Only the URL to the homepage is valid. The second one is no-cache, so you have only 1 URL that can be cached by the crawler. That also means using the crawler makes not much sense if there is only 1 URL to crawl.

    Thread Starter aaronwp03

    (@aaronwp03)

    I do have more than 190 pages but they are pages that is accessed by logged in users only. Is there an automatic way to generate a sitemap to include all of these pages’ urls or I it has to be done one by one?

    I also added this: https://prnt.sc/EqCA80S9e6_s

    I was thinking that doing this will make it automatically crawl all logged in pages. This user role id is the highest user role that has access to all logged in pages.

    I do have more than 190 pages but they are pages that is accessed by logged in users only.

    URls that can only acceessed by logged-in users cannot accessed by the crawler, so there is no way to warmup the cache by the crawler. Regardless, cache for logged-in users is always private. That means every logged-in user has his own cache copy.

    If you want a professional sitemap use any SEO plugin like YOAST SEO (the free one) or Rank Math, but for the crawler use it doesn’t matter, because as said, URLs that can only accessed if page request is possible without login.

    FYI: If you have only 1 URL that can be accessed without login then for search engines you (your page) doesn’t exist. Therefore, you should provide (a lot) content and pages for free (without login).

    Thread Starter aaronwp03

    (@aaronwp03)

    I came from WP Rocket and it has a Preload cache feature which includes all page both public and private.

    The idea is, after purging, WP Rocket will preload pages so that they will be fast just like it was already visited. Because after purging it becomes slow right? Unless you visit it. I asked Hostinger support, they said the equivalent of that with LS is Crawler feature. Is this correct?

    So I have the impression that Crawler works for all pages. Then I saw this on the LS website: https://prnt.sc/Wzr9FAxGg3jA

    That’s why I added a user ID on the role simulation. So to conclude, Crawler only works on public pages and this Role Simulation feature is maybe an outdated feature or maybe something I misinterpreted?


    Thanks for the patience.

Viewing 15 replies - 1 through 15 (of 18 total)
  • The topic ‘LiteSpeed Crawler and Pagespeed Issue’ is closed to new replies.