X-Robots-Tag Header Preventing RSS From Being Indexed

  • Resolved sunkast

    (@sunkast)


    6 years, 5 months ago

    My website has several podcasts with RSS feeds. However Google is unable to index any of these RSS feeds due to the X-Robots-Tag header being set to noindex by Yoast. If I deactivate Yoast, this header tag goes away.

    It is highly important for Google to be able to index RSS feeds now with the way they look for podcasts by indexing them. With the RSS feeds not being indexed by Google, my podcasts will not show up in the new Google Podcasts app.

    Is there a way to prevent Yoast from setting my RSS feeds as noindex with the X-Robots-Tag header?

    The page I need help with: [log in to see the link]

Viewing 11 replies - 1 through 11 (of 11 total)
  • Thread Starter sunkast

    (@sunkast)

    So looking into this, I found this bit of code in class-frontend.php

    	public function noindex_feed() {

		if ( ( is_feed() || is_robots() ) && headers_sent() === false ) {
			header( 'X-Robots-Tag: noindex, follow', true );

			return true;
		}

		return false;
	}

    Is there a way to prevent this header from being set without modifying this file?

    With the way Google is handling podcasts now, having this header negatively impacts RSS feeds that are for podcasts.

    Thanks for reporting the issue. For a temporary work around, please go to PowerPress settings > Feeds tab, select the setting “Do not allow other plugins to modify podcast feeds.” then click save.

    We are looking into tweaking how Yoast’s plugin adds the filters so it does not add them if the page is a feed and PowerPress is enabled.

    Thanks,
    Angelo

    Thread Starter sunkast

    (@sunkast)

    As there has been some confusion in my communication with Yoast via Twitter I want to make clear that the problem with the X-Robots-Tag being set to noindex is being set by Yoast SEO, not by Powerpress or any other plugin.

    Hi @sunkast,

    Thanks for following up.

    We’ve just opened a new GitHub issue with our developers for further investigation. If you’d like to add additional information and follow the progress of the report, please feel free to post a comment to the GitHub issue.

    From my testing it appears the Yoast SEO plugin is adding the tag. A simple fix, do not set the header if the function is_feed() returns true. There should be no need to set noindex for a RSS feed since it is not an HTML page in the first place.

    Hi

    We have added that to the issue report here: https://github.com/Yoast/wordpress-seo/issues/10262. We suggest commenting on the issue report so you may be kept aware of updates. You may need an account in order to post.

    Plugin Support Jerlyn

    (@jerparx)

    Closing thread. Please follow the GH link to know the progress of the issue.

    Apologies as I know this is closed but there doesn’t seem to be a resolution on the GH post or on this thread if you’ve already had the Yoast plugin mess up your header. I have already made the recommended changes to my PowerPress settings. However, the damage is already done. How do you CORRECT the problem now that Yoast made this change already (because unfortunately it seems the recommended workaround only helps people that haven’t installed the plugin yet).

    themealkitreview

    (@themealkitreview)

    joining the discussion, as we don’t see any solution.

    Same issue.

    I use a widget to display an RSS feed of comments, so Yoast SEO decided to add noindex in between <head></head>. Since even the code added to function.php did not roll back the problem, my only fix was to disable Yoast SEO and move on to another plug-in.

    • This reply was modified 5 years, 11 months ago by lokisatyn.

    I have a pre-release version of PowerPress that fixes this issue. If you are using Yoast’s plugin and you are having problems trying to submit your podcast to Google, this should fix the issue. Please contact me cio [at] rawvoice dot com for the link.

    For those who want to wait, this feature will be released PowerPress 7.4.1 in 1-4 weeks, depending on how fast testers get back to us how the fix worked out for them.

    Thanks,
    Angelo

Viewing 11 replies - 1 through 11 (of 11 total)
  • The topic ‘X-Robots-Tag Header Preventing RSS From Being Indexed’ is closed to new replies.