• Resolved nikketrikke

    (@nikketrikke)


    Crawler continue to make BLacklist…
    it works right for 100/200 link then stopped and put all links in blacklist…
    i have plesk server and follow instructions to activate on server…
    now litespeed creates 4 crawlers.. i not why 2 with guest Webp name and 2 guest
    see screenshots
    crawlers
    whitelist
    blocklist
    i’ve tried by cron and to start manually
    have quic cdn active and cloudflare with orange cloud only on www cname
    the connection between server and quic without orange cloud

Viewing 5 replies - 1 through 5 (of 5 total)
  • now litespeed creates 4 crawlers.. i not why 2 with guest Webp name and 2 guest

    I can only answer this. For your backlist issue plugin support must answer it.

    Whenever Guest Mode or webp replacement is enabled then the output code is different. Cache plugin is a fullpage http cache and for caching it generates static files from the dynamically generated URL, but when the output code changes a static file can’t be changed anymore and that’s why each changed code needs its own cached file(URL). Each crawler crawls with different settings and that’s why there is not only 1 crawler but 1 crawler for each setting. Therefore a URL is requested as much as there are settings that causes different output code.

    • This reply was modified 2 years, 7 months ago by serpentdriver.
    Thread Starter nikketrikke

    (@nikketrikke)

    ok so what your suggestion?
    can i delete the crawlers?
    to begin from scratch?
    i’m using a translator fro lanfuages that can change urls…
    so each time need another crawler with other settings?
    SO i not need to delete crwalers but what are the last one to put ON?
    the lower in the list?
    the others must be switched OFF so and never used (anymore)
    have you a link to aguide where to understand how it works.. i read litespeed guide but notunderstand the logic of using crawlers how and when and if need to delete the blacklist each time or need to maintain all blcklist?
    is there a clear guide on how to do?
    thanks

    • This reply was modified 2 years, 7 months ago by nikketrikke.
    Plugin Support qtwrk

    (@qtwrk)

    please provide the report number

    you can get it in toolbox -> report -> click “send to LiteSpeed”

    Thread Starter nikketrikke

    (@nikketrikke)

    MJJPLSFH
    this report number

    Plugin Support qtwrk

    (@qtwrk)

    you should keep crawler as it is , each crawler has a purpose

    you will have different scenarios

    like : crawler for non-webp , crawler for webp , crawler for guest mode user , crawler for guest mode user with webp …etc

    and for blacklist , if I read the screenshot correctly and didn’t have any typo

    picked up 2 URLs , e.g. /da/produkt-tag/iphone-dock-station/ and /product-tag/flip-book-case/ , they are giving 404 error

Viewing 5 replies - 1 through 5 (of 5 total)
  • The topic ‘Crawler continue to make BLacklist…’ is closed to new replies.