Robots.txt has no effect on sitemap

  • Resolved alyxp

    (@alyxp)


    3 years, 6 months ago

    I’ve tried designating specific urls as noindex or none in the robots file but they still show up in the sitemap.

    Trying to avoid having to do it page by page.

    Thanks for your help!

    The page I need help with: [log in to see the link]

Viewing 2 replies - 1 through 2 (of 2 total)
  • Plugin Support devnihil

    (@devnihil)

    @alyxp We’re sorry to hear you are experiencing this issue on your site.

    The robots.txt file does not control whether a URL is included in the sitemap. The directives in robots.txt are only capable of preventing an area of the site from being crawled.

    You can control what content types are included in the sitemap in the settings located in the SEO>Search Appearance area. To customize what types of pages or content are included in the sitemap you would need to perform the following steps:

    ? In WordPress, go to SEO>Search Appearance
    ? Click on the appropriate tab to add or remove certain post types from the search results. Removing a post type from the search results will also remove them from the sitemap.
    ? Click ‘Save Changes’

    We also have additional information on configuring your XML Sitemap located at the following link: https://kb.yoast.com/kb/how-to-customize-the-sitemap-index/

    We also have a guide on how to set a page/post’s meta robots tag to noindex here: https://yoast.com/help/how-do-i-noindex-urls/

    Plugin Support devnihil

    (@devnihil)

    We are going ahead and marking this issue as resolved due to inactivity. If you require any further assistance please create a new issue.

Viewing 2 replies - 1 through 2 (of 2 total)
  • The topic ‘Robots.txt has no effect on sitemap’ is closed to new replies.

Tags