nookyyy

Hi cool,

so what im looking for is a way to crawl a url or even better a sitemap with headers / cookies / useragent etc. ( all data provieded within the action / hook ) . As i have a page with lots of reviews + rating of the products and on each of them theres a list with top products etc. for example, i created a custom logic to purge the relevant pages when i create or update a review. A simple recrawl of the new / edited post wont do in this case.

This way i could add the purged urls into the sitemap or a txt file and then create a cron job to execute a function / php file which then would execute the hook / action to crawl the purged sites and not the whole sitemap of the whole page but only the ones that really need a recrawl / warmup

regards

i have the exact same issue – even when the crawler ran through in every possible mode i always get a miss on first load and after a while again. I logged purges to make sure the test page does not get purged meanwhile aswell – Purge Settings?everything is turned off!

Ok i think i finally found a way – but im not sure if this is how it should work ( feels like some actions are not working or not clearly described in the docs ? )

Whats working now:

// in apply_filters i had to add 'public,no-vary' ( i only had public before ) :

echo apply_filters('litespeed_esi_url', 'custom-block-template-' . $template_id, 'ESI Block ' . $template_id, array('template_id' => $template_id, 'post_id' => $post_id), 'public,no-vary', false, false, true, false);


//in the custom_block_esi_load func:

do_action( 'litespeed_tag_add', 'bricks-template-' . $params['template_id'] );

$esi_blocks_to_purge[] = 'bricks-template-4002'; // Best Mouse — DE
$esi_blocks_to_purge[] = 'bricks-template-4698'; // Best Mouse — EN

foreach ($esi_blocks_to_purge as $block) {
    do_action('litespeed_purge', $block);
}

do_action( ‘litespeed_purge’, ‘ESI’ ); works now aswell – so i guess the no-vary has something to do with this ?

And if i dont want to use a custom tag i would have to target the block:

<!-- X-LiteSpeed-Tag: b79_HTTP.200,b79_ESI,b79_ESI.custom-block-template-4002,b79_bricks-template-4002,b79_ -->

with do_action(‘litespeed_purge’, ‘ESI.custom-block-template-4002’); and not do_action(‘litespeed_purge’, ‘template-4002’); or do_action(‘litespeed_purge_widget’, ‘template-4002’);
this was my mistake

created a file esi.php in root with your code ->

11/08/24 18:24:23.361 [80.139.162.165:40364 1 5Vh] ?? ——GET HTTP/1.1 (HTTPS) /esi.php
11/08/24 18:24:23.361 [80.139.162.165:40364 1 5Vh] Query String:
11/08/24 18:24:23.361 [80.139.162.165:40364 1 5Vh] User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit…
11/08/24 18:24:23.361 [80.139.162.165:40364 1 5Vh] Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
11/08/24 18:24:23.361 [80.139.162.165:40364 1 5Vh] Accept Encoding: br,gzip
11/08/24 18:24:23.361 [80.139.162.165:40364 1 5Vh] Cookie _lscache_vary: admin_bar:1;guest_mode:1;logged-in:1;role:99
11/08/24 18:24:23.361 [80.139.162.165:40364 1 5Vh] X-LSCACHE: true
11/08/24 18:24:23.361 [80.139.162.165:40364 1 5Vh] LSCACHE_VARY_VALUE: +webp
11/08/24 18:24:23.361 [80.139.162.165:40364 1 5Vh] X-LiteSpeed-Purge: public,stale,b79_ESI => LiteSpeed\LSC->send_headers()@624 => WP_Hook->apply_filters(,ARRAY)@324 => WP_Hook->do_action(ARRAY)@348 =>
/var/www/vhosts/nookyyy.com/httpdocs/wp-includes/load.php@517

still the same after i open https://nookyyy.com/esi.php – no esi purge triggered

Just did – same thing nothing happens.

I’ve tested litespeed_add_tag aswell and then purge it with several actions:

if ( is_plugin_active( 'litespeed-cache/litespeed-cache.php' ) ) {
        $debug_string = '';
        foreach ($esi_blocks_to_purge as $block) {
            $debug_string .= 'PURGE: ' . $block . "\n"; 
            do_action('litespeed_purge', $block);
            do_action('litespeed_purge_esi', $block);
            do_action('litespeed_purge_widget', $block);
            
        }

        do_action( 'litespeed_purge', 'ESI' );

        foreach ($esi_page_to_purge as $page) {
            $debug_string .= 'PURGE: ' . $page . "\n";
            do_action( 'litespeed_purge_post', $page );

        } 


        if (!empty($debug_string)) {
            update_field('product_debug', $debug_string, $post_id); 
        }
    }

and the block names are here for example:

$esi_blocks_to_purge[] = 'template-4002'; // Best Mouse — DE
$esi_blocks_to_purge[] = 'template-4698'; // Best Mouse — EN
$esi_blocks_to_purge[] = 'bricks-template-4002'; // Best Mouse — DE
$esi_blocks_to_purge[] = 'bricks-template-4698'; // Best Mouse — EN

and the purge log:

with purge_esi / purge_widget / purge + the blocks in the array

11/08/24 16:00:30.170 [80.139.162.165:58500 1 SVJ] ?? ------POST HTTP/1.1 (HTTPS) /wp-admin/post.php
11/08/24 16:00:30.170 [80.139.162.165:58500 1 SVJ] Query String:
11/08/24 16:00:30.170 [80.139.162.165:58500 1 SVJ] HTTP_REFERER: https://nookyyy.com/wp-admin/post.php?post=3197&actio...
11/08/24 16:00:30.170 [80.139.162.165:58500 1 SVJ] User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit...
11/08/24 16:00:30.170 [80.139.162.165:58500 1 SVJ] Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
11/08/24 16:00:30.170 [80.139.162.165:58500 1 SVJ] Accept Encoding: br,gzip
11/08/24 16:00:30.170 [80.139.162.165:58500 1 SVJ] Cookie _lscache_vary: admin_bar:1;guest_mode:1;logged-in:1;role:99
11/08/24 16:00:30.170 [80.139.162.165:58500 1 SVJ] X-LSCACHE: true
11/08/24 16:00:30.170 [80.139.162.165:58500 1 SVJ] LSCACHE_VARY_VALUE: +webp
11/08/24 16:00:30.170 [80.139.162.165:58500 1 SVJ] X-LiteSpeed-Purge: public,stale,b79_Po.3197,b79_URL./reviews/mouse/vgn/dragonfly-f1-pro-max/,b79_W.recent-posts-1,b79_FD,b79_REST,b79_template-4002,b79_ESI.template-4002,b79_template-4698,b79_ESI.template-4698,b79_bricks-template-4002,b79_ESI.bricks-template-4002,b79_bricks-template-4698,b79_ESI.bricks-template-4698,b79_Po.4050,b79_URL./reviews/mouse/,b79_Po.4708,b79_URL./en/reviews/mouse/ => LiteSpeed\LSC->send_headers()@624 => WP_Hook->apply_filters(,ARRAY)@324 => WP_Hook->do_action(ARRAY)@348 =>
/var/www/vhosts/nookyyy.com/httpdocs/wp-includes/load.php@517

with purge 'ESI'

11/08/24 16:04:05.073 [80.139.162.165:39396 1 O2m] ?? ------POST HTTP/1.1 (HTTPS) /wp-admin/post.php
11/08/24 16:04:05.073 [80.139.162.165:39396 1 O2m] Query String:
11/08/24 16:04:05.073 [80.139.162.165:39396 1 O2m] HTTP_REFERER: https://nookyyy.com/wp-admin/post.php?post=3197&actio...
11/08/24 16:04:05.073 [80.139.162.165:39396 1 O2m] User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit...
11/08/24 16:04:05.073 [80.139.162.165:39396 1 O2m] Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
11/08/24 16:04:05.073 [80.139.162.165:39396 1 O2m] Accept Encoding: br,gzip
11/08/24 16:04:05.073 [80.139.162.165:39396 1 O2m] Cookie _lscache_vary: admin_bar:1;guest_mode:1;logged-in:1;role:99
11/08/24 16:04:05.073 [80.139.162.165:39396 1 O2m] X-LSCACHE: true
11/08/24 16:04:05.073 [80.139.162.165:39396 1 O2m] LSCACHE_VARY_VALUE: +webp
11/08/24 16:04:05.073 [80.139.162.165:39396 1 O2m] X-LiteSpeed-Purge: public,stale,b79_Po.3197,b79_URL./reviews/mouse/vgn/dragonfly-f1-pro-max/,b79_W.recent-posts-1,b79_FD,b79_REST,b79_ESI,b79_Po.4050,b79_URL./reviews/mouse/,b79_Po.4708,b79_URL./en/reviews/mouse/ => LiteSpeed\LSC->send_headers()@624 => WP_Hook->apply_filters(,ARRAY)@324 => WP_Hook->do_action(ARRAY)@348 =>
/var/www/vhosts/nookyyy.com/httpdocs/wp-includes/load.php@517

This is one block for example i want to purge:

<!-- Block cached by LiteSpeed Cache 6.5.2 on 2024-11-08 14:16:36 -->
<!-- X-LiteSpeed-Cache-Control: public,max-age=120,esi=on -->
<!-- X-LiteSpeed-Tag: b79_HTTP.200,b79_ESI,b79_ESI.custom-block-template-4002,b79_W.bricks-template-4002,b79_ -->

which i create here:

function custom_block_esi_load($params) {

    do_action( 'litespeed_control_set_ttl', 120 );
    do_action( 'litespeed_tag_add_widget', 'bricks-template-' . $params['template_id'] );
       
    //block content
}

and here is the filter for the esi_url

echo apply_filters('litespeed_esi_url', 'custom-block-template-' . $template_id, 'ESI Block ' . $template_id, array('template_id' => $template_id, 'post_id' => $post_id), 'public', false, false, true, false);

I’ve tested the actions described here (https://docs.litespeedtech.com/lscache/lscwp/api/#purge-a-cache-tag) but nothing works – the automatic purge of a page by id works ( when theres a new product for example i have to purge two sites but on the others like frontpage i only need to purge the esi block )

Hi any update yet ?

LOL – i randomly tried again, and it worked now but i sent it blank without an info text or 1 time login:

Report number: XHRVHDPE

Report date: 09/27/2024 21:04:55

Sure i tried again but still same error.

And when i want to try a second time i get: Cloud Error: Please try after 4m 46s for service wp/report.

I have sent a report from another wp instance on the same server, which worked flawless, so there has to be some issue with the wp system on that domain only or something else ( maybe this info helps )

array(6) {
  ["headers"]=>
  object(WpOrg\Requests\Utility\CaseInsensitiveDictionary)#10796 (1) {
    ["data":protected]=>
    array(7) {
      ["content-type"]=>
      string(24) "text/html; charset=UTF-8"
      ["content-length"]=>
      string(2) "47"
      ["content-encoding"]=>
      string(2) "br"
      ["vary"]=>
      string(15) "Accept-Encoding"
      ["date"]=>
      string(29) "Thu, 26 Sep 2024 21:21:23 GMT"
      ["server"]=>
      string(9) "LiteSpeed"
      ["alt-svc"]=>
      string(167) "h3=":443"; ma=2592000, h3-29=":443"; ma=2592000, h3-Q050=":443"; ma=2592000, h3-Q046=":443"; ma=2592000, h3-Q043=":443"; ma=2592000, quic=":443"; ma=2592000; v="43,46""
    }
  }
  ["body"]=>
  string(48) "{"_res":"err","_msg":"data format error no env"}"
  ["response"]=>
  array(2) {
    ["code"]=>
    int(200)
    ["message"]=>
    string(2) "OK"
  }
  ["cookies"]=>
  array(0) {
  }
  ["filename"]=>
  NULL
  ["http_response"]=>
  object(WP_HTTP_Requests_Response)#10795 (5) {
    ["data"]=>
    NULL
    ["headers"]=>
    NULL
    ["status"]=>
    NULL
    ["response":protected]=>
    object(WpOrg\Requests\Response)#10797 (10) {
      ["body"]=>
      string(48) "{"_res":"err","_msg":"data format error no env"}"
      ["raw"]=>
      string(425) "HTTP/1.1 200 OK
Connection: close
content-type: text/html; charset=UTF-8
content-length: 47
content-encoding: br
vary: Accept-Encoding
date: Thu, 26 Sep 2024 21:21:23 GMT
server: LiteSpeed
alt-svc: h3=":443"; ma=2592000, h3-29=":443"; ma=2592000, h3-Q050=":443"; ma=2592000, h3-Q046=":443"; ma=2592000, h3-Q043=":443"; ma=2592000, quic=":443"; ma=2592000; v="43,46"

{"_res":"err","_msg":"data format error no env"}"
      ["headers"]=>
      object(WpOrg\Requests\Response\Headers)#10798 (1) {
        ["data":protected]=>
        array(7) {
          ["content-type"]=>
          array(1) {
            [0]=>
            string(24) "text/html; charset=UTF-8"
          }
          ["content-length"]=>
          array(1) {
            [0]=>
            string(2) "47"
          }
          ["content-encoding"]=>
          array(1) {
            [0]=>
            string(2) "br"
          }
          ["vary"]=>
          array(1) {
            [0]=>
            string(15) "Accept-Encoding"
          }
          ["date"]=>
          array(1) {
            [0]=>
            string(29) "Thu, 26 Sep 2024 21:21:23 GMT"
          }
          ["server"]=>
          array(1) {
            [0]=>
            string(9) "LiteSpeed"
          }
          ["alt-svc"]=>
          array(1) {
            [0]=>
            string(167) "h3=":443"; ma=2592000, h3-29=":443"; ma=2592000, h3-Q050=":443"; ma=2592000, h3-Q046=":443"; ma=2592000, h3-Q043=":443"; ma=2592000, quic=":443"; ma=2592000; v="43,46""
          }
        }
      }
      ["status_code"]=>
      int(200)
      ["protocol_version"]=>
      float(1.1)
      ["success"]=>
      bool(true)
      ["redirects"]=>
      int(0)
      ["url"]=>
      string(32) "https://api.quic.cloud/wp/report"
      ["history"]=>
      array(0) {
      }
      ["cookies"]=>
      object(WpOrg\Requests\Cookie\Jar)#10794 (1) {
        ["cookies":protected]=>
        array(0) {
        }
      }
    }
    ["filename":protected]=>
    NULL
  }
}

?hm i get this error on sending to litespeed:

Failed to request via WordPress: cURL error 28: Operation timed out after 15001 milliseconds with 0 bytes received [server] https://api.quic.cloud [service] wp/report

Hi Ticket ID is #980556

regards

Hey just to clarify via mail or the litespeed support ticket system (we own the licence / server)

And in case you missed my other question here once again ??

Furthermore i got a question, as these 2 blocks basicly have the same content ( apart from the translations ) is it possible to just have 1 ESI block with some sort of unique identifier to distinguish between the languages. Maybe a tag like custom-esi-“ID”-“LANGPARAMTER”. If yes how would this have to be implemented? And how can i then purge that specific block with the associated unique id / tag if this is possible.

regards

Hey, the content of that css file (wow i should have had a look inside already – yikez ^^):

/* Failed to generate CCSS */

According to the filesystem log it was generated 22.09.2024 – 06:10:13 CEST

2nd Question:

For example i have /home /reviews /contact etc. but all these pages are translated ( right now only in english but other languages are planned – /en/home /en/reviews etc ) – every page is of post-type page and has exactly the same structure / content ( just translated ). But for every page a unique ucss is generated. Its not a massive deal to generate a ucss for every page + translation but it would be nice if theres a way to use only 1 ucss for a page + its translations ( /home + /en/home = same ucss and so on )

regards

The issue happened yesterday evening once again – im still unable to tell what triggers it – but this morning its gone once again. My guess is, that for some reason the “old” nonce ( they have a ttl of 12h right ? ) was somehow cached / not updated and then the error happens.

If i find out how to reproduce it i will update you