How to prevent scanning sitemap XML files

  • Resolved idovecer

    (@idovecer)


    1 year, 9 months ago

    Why does W3 Total Cache constantly scan/GET all XML files?
    I’m only worried because during these scanning periods, my server processor time increases a lot and loads the server, considering the large number of posts involved in this wordpress site.

    According to the log, I noticed that this often happens exactly every 20 minutes (1200 seconds).

    Is it a good idea to turn it off?
    What is it for and why does it work?
    And how to turn it off?

    Thank you.

    *.*.*.* - - [05/Feb/2023:16:03:03 +0100] "GET /category-sitemap.xml HTTP/1.1" 200 2392 "https://www.some-domain.com/category-sitemap.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:03 +0100] "GET /post_tag-sitemap.xml HTTP/1.1" 200 16846 "https://www.some-domain.com/post_tag-sitemap.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:04 +0100] "GET /post_tag-sitemap2.xml HTTP/1.1" 200 16447 "https://www.some-domain.com/post_tag-sitemap2.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:04 +0100] "GET /post_tag-sitemap3.xml HTTP/1.1" 200 16151 "https://www.some-domain.com/post_tag-sitemap3.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:04 +0100] "GET /post_tag-sitemap4.xml HTTP/1.1" 200 15761 "https://www.some-domain.com/post_tag-sitemap4.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:04 +0100] "GET /post_tag-sitemap5.xml HTTP/1.1" 200 16107 "https://www.some-domain.com/post_tag-sitemap5.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:04 +0100] "GET /post_tag-sitemap6.xml HTTP/1.1" 200 15519 "https://www.some-domain.com/post_tag-sitemap6.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:05 +0100] "GET /post_tag-sitemap7.xml HTTP/1.1" 200 16512 "https://www.some-domain.com/post_tag-sitemap7.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:05 +0100] "GET /post_tag-sitemap8.xml HTTP/1.1" 200 16253 "https://www.some-domain.com/post_tag-sitemap8.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:05 +0100] "GET /post_tag-sitemap9.xml HTTP/1.1" 200 15504 "https://www.some-domain.com/post_tag-sitemap9.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:05 +0100] "GET /post_tag-sitemap10.xml HTTP/1.1" 200 16479 "https://www.some-domain.com/post_tag-sitemap10.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:05 +0100] "GET /post_tag-sitemap11.xml HTTP/1.1" 200 16227 "https://www.some-domain.com/post_tag-sitemap11.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:05 +0100] "GET /post_tag-sitemap12.xml HTTP/1.1" 200 16349 "https://www.some-domain.com/post_tag-sitemap12.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:05 +0100] "GET /post_tag-sitemap13.xml HTTP/1.1" 200 16316 "https://www.some-domain.com/post_tag-sitemap13.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:06 +0100] "GET /post_tag-sitemap14.xml HTTP/1.1" 200 16147 "https://www.some-domain.com/post_tag-sitemap14.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:06 +0100] "GET /post_tag-sitemap15.xml HTTP/1.1" 200 15786 "https://www.some-domain.com/post_tag-sitemap15.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:06 +0100] "GET /post_tag-sitemap16.xml HTTP/1.1" 200 16509 "https://www.some-domain.com/post_tag-sitemap16.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:06 +0100] "GET /post_tag-sitemap17.xml HTTP/1.1" 200 15539 "https://www.some-domain.com/post_tag-sitemap17.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:06 +0100] "GET /post_tag-sitemap18.xml HTTP/1.1" 200 16490 "https://www.some-domain.com/post_tag-sitemap18.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:06 +0100] "GET /post_tag-sitemap19.xml HTTP/1.1" 200 15804 "https://www.some-domain.com/post_tag-sitemap19.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:07 +0100] "GET /post_tag-sitemap20.xml HTTP/1.1" 200 16312 "https://www.some-domain.com/post_tag-sitemap20.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:07 +0100] "GET /post_tag-sitemap21.xml HTTP/1.1" 200 16065 "https://www.some-domain.com/post_tag-sitemap21.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:07 +0100] "GET /post_tag-sitemap22.xml HTTP/1.1" 200 15978 "https://www.some-domain.com/post_tag-sitemap22.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:07 +0100] "GET /post_tag-sitemap23.xml HTTP/1.1" 200 15747 "https://www.some-domain.com/post_tag-sitemap23.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:07 +0100] "GET /post_tag-sitemap24.xml HTTP/1.1" 200 15663 "https://www.some-domain.com/post_tag-sitemap24.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:07 +0100] "GET /post_tag-sitemap25.xml HTTP/1.1" 200 15484 "https://www.some-domain.com/post_tag-sitemap25.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:08 +0100] "GET /post_tag-sitemap26.xml HTTP/1.1" 200 16695 "https://www.some-domain.com/post_tag-sitemap26.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:08 +0100] "GET /post_tag-sitemap27.xml HTTP/1.1" 200 16172 "https://www.some-domain.com/post_tag-sitemap27.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:08 +0100] "GET /post_tag-sitemap28.xml HTTP/1.1" 200 16888 "https://www.some-domain.com/post_tag-sitemap28.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:08 +0100] "GET /post_tag-sitemap29.xml HTTP/1.1" 200 16820 "https://www.some-domain.com/post_tag-sitemap29.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:08 +0100] "GET /post_tag-sitemap30.xml HTTP/1.1" 200 15747 "https://www.some-domain.com/post_tag-sitemap30.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:09 +0100] "GET /post_tag-sitemap31.xml HTTP/1.1" 200 16502 "https://www.some-domain.com/post_tag-sitemap31.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:09 +0100] "GET /post_tag-sitemap32.xml HTTP/1.1" 200 16113 "https://www.some-domain.com/post_tag-sitemap32.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:09 +0100] "GET /post_tag-sitemap33.xml HTTP/1.1" 200 16012 "https://www.some-domain.com/post_tag-sitemap33.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:09 +0100] "GET /post_tag-sitemap34.xml HTTP/1.1" 200 15961 "https://www.some-domain.com/post_tag-sitemap34.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:09 +0100] "GET /post_tag-sitemap35.xml HTTP/1.1" 200 16484 "https://www.some-domain.com/post_tag-sitemap35.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:10 +0100] "GET /post_tag-sitemap36.xml HTTP/1.1" 200 16881 "https://www.some-domain.com/post_tag-sitemap36.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:10 +0100] "GET /post_tag-sitemap37.xml HTTP/1.1" 200 16930 "https://www.some-domain.com/post_tag-sitemap37.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:10 +0100] "GET /post_tag-sitemap38.xml HTTP/1.1" 200 16344 "https://www.some-domain.com/post_tag-sitemap38.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:10 +0100] "GET /post_tag-sitemap39.xml HTTP/1.1" 200 16277 "https://www.some-domain.com/post_tag-sitemap39.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:10 +0100] "GET /post_tag-sitemap40.xml HTTP/1.1" 200 15692 "https://www.some-domain.com/post_tag-sitemap40.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:11 +0100] "GET /post_tag-sitemap41.xml HTTP/1.1" 200 16825 "https://www.some-domain.com/post_tag-sitemap41.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:11 +0100] "GET /post_tag-sitemap42.xml HTTP/1.1" 200 16151 "https://www.some-domain.com/post_tag-sitemap42.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:11 +0100] "GET /post_tag-sitemap43.xml HTTP/1.1" 200 16362 "https://www.some-domain.com/post_tag-sitemap43.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:11 +0100] "GET /post_tag-sitemap44.xml HTTP/1.1" 200 16042 "https://www.some-domain.com/post_tag-sitemap44.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:11 +0100] "GET /post_tag-sitemap45.xml HTTP/1.1" 200 16622 "https://www.some-domain.com/post_tag-sitemap45.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:12 +0100] "GET /post_tag-sitemap46.xml HTTP/1.1" 200 16117 "https://www.some-domain.com/post_tag-sitemap46.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:12 +0100] "GET /post_tag-sitemap47.xml HTTP/1.1" 200 16848 "https://www.some-domain.com/post_tag-sitemap47.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:12 +0100] "GET /post_tag-sitemap48.xml HTTP/1.1" 200 17046 "https://www.some-domain.com/post_tag-sitemap48.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:12 +0100] "GET /post_tag-sitemap49.xml HTTP/1.1" 200 16262 "https://www.some-domain.com/post_tag-sitemap49.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:13 +0100] "GET /post_tag-sitemap50.xml HTTP/1.1" 200 16029 "https://www.some-domain.com/post_tag-sitemap50.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:13 +0100] "GET /post_tag-sitemap51.xml HTTP/1.1" 200 17054 "https://www.some-domain.com/post_tag-sitemap51.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:13 +0100] "GET /post_tag-sitemap52.xml HTTP/1.1" 200 16055 "https://www.some-domain.com/post_tag-sitemap52.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:13 +0100] "GET /post_tag-sitemap53.xml HTTP/1.1" 200 11808 "https://www.some-domain.com/post_tag-sitemap53.xml" "W3 Total Cache"
*.*.*.* - - [05/Feb/2023:16:03:15 +0100] "GET /author-sitemap.xml HTTP/1.1" 200 912 "https://www.some-domain.com/author-sitemap.xml" "W3 Total Cache
    • This topic was modified 1 year, 9 months ago by idovecer.
Viewing 1 replies (of 1 total)
  • Plugin Contributor Marko Vasiljevic

    (@vmarko)

    Hello @idovecer

    Thank you for reaching out and I am happy to assist you with this.

    Can you please check and confirm if the Cache Preload is enabled in Performance>Page Cache section and have you added the sitemap URL there?

    Thanks!

Viewing 1 replies (of 1 total)
  • The topic ‘How to prevent scanning sitemap XML files’ is closed to new replies.