• Resolved narelly

    (@narelly)


    Hi. Is there a way in Yoast SEO to exclude an entire folder (non WordPress folder) from robots.txt?

    The page I need help with: [log in to see the link]

Viewing 4 replies - 1 through 4 (of 4 total)
  • Plugin Support devnihil

    (@devnihil)

    @narelly Yes you can disallow an entire folder by using a Disallow directive in the robots.txt file. For example if you wanted for all user agents (Googlebot, Bing, etc) to disallow a folder located at the root of your site name ‘Images’, you would include the following directive in your robots.txt:

    User-agent: * 
    Disallow: /Images

    Also, you can find out more about the syntax of robots.txt in our guide at https://yoast.com/ultimate-guide-robots-txt/ as well as Google’s documentation here: https://developers.google.com/search/reference/robots_txt

    You can edit the robots.txt file using the Yoast SEO plugin by performing the following steps:

      In WordPress, go to SEO>Tools
      Click on ‘File Editor’
      Under ‘robots.txt’ beneath ‘Edit the content of your robots.txt:’ is a field from which you can directly edit your robots.txt file
      After making any changes to the file, click ‘Save changes to robots.txt’

    We also have more information on this and editing the robots.txt file here: https://kb.yoast.com/kb/how-to-edit-robots-txt-through-yoast-seo/

    Thread Starter narelly

    (@narelly)

    Thank you devnihil, I manage to do it, no problem ?? How can I exclude the same folder from the sitemap?

    Pcosta88

    (@pcosta88)

    @narelly

    We looked at https://apponyikiskastely.hu/sitemap_index.xml and we are not seeing an image folder on the sitemap. Did you resolve this issue?

    Thread Starter narelly

    (@narelly)

    The folder Images was just an example in devnihil’s post ?? But in the meantime I managed to resolve the issue, thank you.

Viewing 4 replies - 1 through 4 (of 4 total)
  • The topic ‘Exclude entire folder from robots.txt’ is closed to new replies.