‘noindex’ detected in ‘X-Robots-Tag’ http header for google search console

@g360riley Just to confirm, is this notification from Google Search Console restricted to your site’s sitemap?

If so, the notification is not incorrect, but the sitemap should include a noindex tag, as its not meant to be part of Google’s index of what is returned for a user’s search as it’s not useful for them.

As for the reason you aren’t seeing the noindex tag, this is because it’s not output in the source code of the file, but rather in the header as it’s added via the X-Robots tag. You can see it output though if you check the sitemap via a terminal with cURL:

curl -I https://allthingshentai.ddns.net/sitemap_index.xml
HTTP/1.1 200 OK
Date: Thu, 05 May 2022 00:14:05 GMT
Server: Apache/2.4.46 (Win64) OpenSSL/1.1.1g PHP/7.3.21
X-Powered-By: PHP/7.3.21
X-Robots-Tag: noindex, follow
Content-Type: text/xml; charset=UTF-8

@devnihil so yes, google search console throws this error when I try to submit my site map. Is there anything that I can do to fix this issue or will my sitemap on google console be forever broken? I’ve let google try to process it for a few months but the error has never gone away.

Hello G360riley,

It is normal for your sitemap to have that; your sitemap is not broken but loads perfectly fine.

The noindex X-Robots-Tag is expected. We want Google and other search engines to crawl the XML Sitemaps and detect the links inside them, but we do not want the XML Sitemaps to be shown in the search results; they’re not meant to be used by regular users, only crawlers.

I hope that clears your doubts

We are going ahead and marking this issue as resolved due to inactivity. If you require any further assistance please create a new issue.

I have exactly the same problem
1500 of my site’s posts are Noindexed by X-Robot-Tag In Webmaster
In none of these posts, the Noindex tag is used, and all of them are index and follow, and I don’t know where these errors come from.
Help me please
An example of a Noindex link
sabalanmusic

Hello @sabalanmusic ! Good day!

Did you solve this problem? I’ve checked several posts on the Internet (including here) and I couldn’t find a solution.

This statement by @maybellyne “The noindex X-Robots-Tag is expected. We want Google and other search engines to crawl the XML Sitemaps and detect the links inside them, but we do not want the XML Sitemaps to be shown in the search results; they’re not meant to be used by regular users, only crawlers.” makes sense. However, now, different than 1 week ago, when I check to see if my website is indexed (site: pacmigas.com) is not possible to find it.

There is something wrong with the plugin. I was wondering if the best way to (try to) solve this issue is change it for its main competitor.

I appreciate any help.

Thank you in advance.

Best regards, Adriano Andrade.

Hey @adri1001 this issue was never fixed for me. Eventually google will index the page but it might take a while. I had to wait almost 6 months for a (very severe) case of this to finally be indexed. Although in google search console it will say it can’t find it because of the noindex tag, for some reason it actually can. Mine still says the same thing but google indexes pages like normal.

Hello @g360riley !

Thank you very much for your prompt reply. Today early I talked to the Support of my hosting (Siteground) and they also couldn’t find any issue. So, my plan of action is to do what you’ve done: study, learn and wait patiently. Again, thank you.

Best regards, Adriano Andrade.

• This reply was modified 2 years, 4 months ago by adri1001.