W3TC & YOAST sitemap

  • Hi,
    I was controlling some parameters of my website, and found that the Yoast sitemap is been cached no matter which settings I set within W3TC control panel. URL mysweetlaura.it/sitemap.xml is not being cached, but Yoast redirects it to mysweetlaura.it/sitemap_index.xml and this final location is being cached. Despite excluding this with regex in “never cache this pages” in page cache, browser cache, minify, etc. Primed cache is not enabled, nor database cache. The only way I found to prevent this situation is disabling browser cache, but this obviously, won’t be the case of having a caching plugin.

    I will really appreciate any helpful information on this. I searched for it on Google and within this forum to no avail.

    This is what I see on tools.seobook.com/server-header-checker (browser cache enabled):

    *****************************

    Requesting https://www.mysweetlaura.it/sitemap.xml

SERVER RESPONSE: HTTP/1.1 301 Moved Permanently
Date:
    Mon, 11 Jun 2018 22:43:35 GMT
Server:
    Apache
X-Powered-By:
    W3 Total Cache/0.9.7
Expires:
    Wed, 11 Jan 1984 05:00:00 GMT
Cache-Control:
    no-cache, must-revalidate, max-age=0
Upgrade:
    h2,h2c
Connection:
    Upgrade, close
Location:
    https://www.mysweetlaura.it/sitemap_index.xml
Vary:
    User-Agent
Referrer-Policy:
Content-Type:
    text/html; charset=UTF-8 

Redirecting to: https://www.mysweetlaura.it/sitemap_index.xml

SERVER RESPONSE: HTTP/1.1 200 OK
Date:
    Mon, 11 Jun 2018 22:43:36 GMT
Server:
    Apache
X-Powered-By:
    W3 Total Cache/0.9.7
X-Robots-Tag:
    noindex, follow
Upgrade:
    h2,h2c
Connection:
    Upgrade, close
Cache-Control:
    max-age=3600
Expires:
    Mon, 11 Jun 2018 23:43:36 GMT
Vary:
    User-Agent
Referrer-Policy:
Content-Type:
    text/xml; charset=UTF-8

    *****************************

    FWIW, same tool, browser cache disabled:

    *****************************

    Requesting https://www.mysweetlaura.it/sitemap.xml

SERVER RESPONSE: HTTP/1.1 301 Moved Permanently
Date:
    Mon, 11 Jun 2018 23:28:55 GMT
Server:
    Apache
Expires:
    Wed, 11 Jan 1984 05:00:00 GMT
Cache-Control:
    no-cache, must-revalidate, max-age=0
Upgrade:
    h2,h2c
Connection:
    Upgrade, close
Location:
    https://www.mysweetlaura.it/sitemap_index.xml
Vary:
    User-Agent
Content-Type:
    text/html; charset=UTF-8 

Redirecting to: https://www.mysweetlaura.it/sitemap_index.xml

SERVER RESPONSE: HTTP/1.1 200 OK
Date:
    Mon, 11 Jun 2018 23:28:55 GMT
Server:
    Apache
X-Robots-Tag:
    noindex, follow
Upgrade:
    h2,h2c
Connection:
    Upgrade, close
Vary:
    User-Agent
Content-Type:
    text/xml; charset=UTF-8

    The page I need help with: [log in to see the link]

Viewing 4 replies - 1 through 4 (of 4 total)
  • Plugin Contributor gidomanders

    (@gidomanders)

    If you go to Performance => Browser Cache and add [a-z0-9_\-]*sitemap[a-z0-9_\-]*\.(xml|xsl|html)(\.gz)? to the Prevent caching exception list: field, does it still cache the sitemap?

    As I can see from the response, the sitemaps will be cached for an hour, so I think it won’t be that much of a problem, at least not for Google.

    Thread Starter supermemo

    (@supermemo)

    I tried that already. Else, I set that in all Page cache section, Minify section and Browser cache (Do not process 404 errors… 404 error exception list). Moreover, if in Browser cache section, HTML & XML section, I change “expires header lifetime” to an arbitrary value (e.g. 50 sec) it is set correctly. This demonstrate that I have some degree of control over the settings, but cannot prevent caching at all, nor I can set the e-tag (solely to say that maybe there is something wrong with the parameters settings). Cache control policy is set to “cache with max-age (“public, max-age=EXPIRES_SECONDS”)

    SERVER RESPONSE: HTTP/1.1 200 OK
Date:
    Tue, 12 Jun 2018 18:10:18 GMT
Server:
    Apache
X-Powered-By:
    W3 Total Cache/0.9.7
X-Robots-Tag:
    noindex, follow
Upgrade:
    h2,h2c
Connection:
    Upgrade, close
Cache-Control:
    max-age=50
Expires:
    Tue, 12 Jun 2018 18:11:08 GMT
Vary:
    User-Agent
Referrer-Policy:
Content-Type:
    text/xml; charset=UTF-8
    • This reply was modified 6 years, 9 months ago by supermemo.
    Thread Starter supermemo

    (@supermemo)

    Maybe I found the culprit:
    Browser cache section, General subsection:
    * Set Last-Modified header
    * Set expires header
    * Set cache control header
    * Set entity tag (ETag)
    * Set W3 Total Cache header
    * Enable HTTP (gzip) compression
    * Do not process 404 errors for static objects with WordPress
    These were all set. Toggled off all of them, and then toggled on one by one.
    The only one that activates the sitemap cache is “Set expires header”, regardless of the others’ status. I don’t think that this is the desired outcome of this setting, but I maybe wrong…

    Plugin Contributor gidomanders

    (@gidomanders)

    The Expires header forces your browser to cache the response for a certain amount of seconds. As I explained in my previous reply, your settings make the browser cache the sitemap for an hour. Normally that won’t be an issue, at least not for Google as they don’t use those headers when crawling the sitemap.

Viewing 4 replies - 1 through 4 (of 4 total)
  • The topic ‘W3TC & YOAST sitemap’ is closed to new replies.