• Resolved ashaduri

    (@ashaduri)


    Hi,

    Today Google informed me that my sitemap URL is marked “noindex”. If I understand the error correctly, Google won’t crawl the sitemap because of this. Indeed, upon inspecting the HTTP headers, I see x-robots-tag: noindex.

    The sitemap is virtually generated by the Google XML Sitemaps plugin.

    How can I remove this header?

    Thanks,
    Alexander

    The page I need help with: [log in to see the link]

Viewing 14 replies - 16 through 29 (of 29 total)
  • if you have access to the files go to the file:
    wp-content/plugins/google-sitemap-generator/sitepmap.xml
    and change the line
    <meta name=”robots” content=”noindex,follow” />
    to
    <meta name=”robots” content=”index,follow” />

    not an ideal solution and will prob be overwritten once the plugin is updated but it is what it is. plugin author should differentiate between the html and xml outputs. should be fine if the html file is indexed.

    i had the same issue i followed the following
    if you have access to the files go to the file:
    wp-content/plugins/google-sitemap-generator/sitepmap.xml
    and change the line
    <meta name=”robots” content=”noindex,follow” />
    to
    <meta name=”robots” content=”index,follow” />

    not an ideal solution and will prob be overwritten once the plugin is updated but it is what it is. plugin author should differentiate between the html and xml outputs. should be fine if the html file is indexed.

    i had the same issue for one client Pinoy Replays but solved by following
    if you have access to the files go to the file:
    wp-content/plugins/google-sitemap-generator/sitepmap.xml
    and change the line
    <meta name=”robots” content=”noindex,follow” />
    to
    <meta name=”robots” content=”index,follow” />

    not an ideal solution and will prob be overwritten once the plugin is updated but it is what it is. plugin author should differentiate between the html and xml outputs. should be fine if the html file is indexed.

    • This reply was modified 5 years, 11 months ago by tom81812.

    @s4kashi I see that you switched over to Yoast SEO, and are using their Sitemap function. How is that working for you?

    There may be some additional tips on this resource doc, which covers the “Submitted URL” issue people encounter in the new-ish Google Search Console.

    I’ve been looking at the entries in this thread because my sitemap.xml file wasn’t being rendered. it was coming up blank and Google wasn’t able to fetch it.

    I was also looking at this problem as a wider review because I was hacked with what Google called a “content injection”. Pharma junk was added to my pages – not so it was visible to viewers, but visible in a Google fetch of the page. I have been reviewing everything and looking at all problems. Uninstalling and reinstalling plugins. And noticed in that review this issue with the sitemap.xml files.

    Turns out… its part of the same hack. And it’s not a hack of the plugin. Its a hack of the rendering of .xml pages on the site. So even when I installed a different plugin that rendered the .xml page, it still was blank. The point for the hacker is that then Google crawled my site anyways and found their extra content, that were pages that wouldn’t have been indexed in my sitemap.xml file. hmmmm….

    The main thing for the rendering of the sitemap.xml file was that they added stuff to both my .htaccess file and wp-config file that are stored in the ./html top level directory. So outside of the plugin. They added on routines. When I removed these, the sitemap.xml file rendered properly. (I’ve also removed other stuff, but this was key for the plugin to work properly… it’s been a horrible job to clean everything!!!)

    Hope this helps people… though I hope you weren’t hacked!

    cheers!

    Anonymous User 14050981

    (@anonymized-14050981)

    don’t use generated sitemaps from wordpress plugins go to this link to generate your sitemap https://www.xml-sitemaps.com/ also you will need to hire a professional coder from codeable to remove the Submitted URL marked ‘noindex from your website

    Anonymous User 14050981

    (@anonymized-14050981)

    the yoast seo wordpress plugin can generate a noindex url in your website remove the plugin to fix the problem also check your robots.txt to see if you allow or disallow certain rules for your website and you should use pro sitemaps to generate your sitemap also for your website

    Anonymous User 14050981

    (@anonymized-14050981)

    I found a better way just install and activate the all in one seo pack wordpress plugin
    then uncheck all the noindex in the plugin settings also you can install and activate the xml sitemaps plugin by Arne Brachhold to generate your site map for your wordpress website

    Anonymous User 14050981

    (@anonymized-14050981)

    A even better way is to just install and activate the all in one seo pack wordpress plugin then uncheck all the noindex in the plugin settings also you can activate the sitemap from the all in one seo pack under the features tab and copy and paste the info
    in your robots.txt file

    I’m having this problem

    ‘noindex’ detected in ‘X-Robots-Tag’ http header

    This is from the google search console on live URL test in ‘URL Inspection’ and in the sitemaps section ‘Couldn’t Fetch’.

    Have been trying for months!

    A bit poor that google’s own plugin doesn’t work with google search console!
    Have tried YOAST but although the url is different (sitemap_indx.xml) result is the same.

    Is there a definitive answer for this?

    Anonymous User 14050981

    (@anonymized-14050981)

    Hi use the all in one seo pack and uncheck noindex on each in the general settings also generate your robots.txt file and sitemap under the feature manager tab with the all in one seo pack don’t forget to copy and paste the robots.txt info into your robots.txt file which includes the sitemap link

    @sebdinatale

    So I did that and to be sure deleted all cahces and disabled all of the other plugins and..

    The sitemap has been submitted ‘Success’ but…

    When I fo a ‘Live Test’ in the URL Inspection. .

    ?
    Coverage
    Excluded by ‘noindex’ tag
    Discovery
    Sitemaps
    N/A
    Referring page
    None detected
    URL might be known from other sources that are currently not reported
    Crawl
    Last crawl
    Oct 20, 2018, 5:08:11 PM
    Crawled as
    Googlebot desktop
    Crawl allowed?
    Yes
    Page fetch
    Successful
    Indexing
    Indexing allowed?
    ?
    No: ‘noindex’ detected in ‘X-Robots-Tag’ http header
    User-declared canonical
    N/A
    Google-selected canonical
    ?
    N/A

    • This reply was modified 5 years, 2 months ago by timxjr1300.

    So..

    Disabled all plugins.
    Tested ‘sitemap.xml’ with search console ‘live’ test
    All worked.

    Enable Yoast XML Sitemaps – live test ‘blocked by noindex’
    Enabled google sitemap plug in – live test – ‘blocked by noindex’

    Disabled google plugin, switched off Yoast xml sitemaps.. all works fine.

    Will just have to accept that I don’t have very much control over what is and what isn’t indexed.

    Will also have to accept that none of these plugins work as advertised. Very disappointed.

Viewing 14 replies - 16 through 29 (of 29 total)
  • The topic ‘Submitted URL marked ‘noindex’’ is closed to new replies.