Robots.txt File Question

  • Can you assist me with correctly writing a robots.txt file

    My robots.txt file will be loaded on the root https://www.abogados.us/robots.txt

    Our programs Joomla and WordPress will be loaded on seperate folders.

    1) How can I communicate via the robots.txt file that my disallow items will be located in respective folders

    2) I have several sites, Can the robots.txt file be written in a way that can cover the folder names with a wildcard effect?

    Sincerely,

    Geoffrey Gonzalez
    Tel: 203-746-7589

    User-agent: * (THESE ARE WORDPRESS ITEMS)
    User-agent: *
    Disallow: /*/email/$
    Disallow: /*/print/$
    Disallow: /*/feed/$
    Disallow: /*/feed/rss/$
    Disallow: /*/trackback/$
    Disallow: /wp-
    Disallow: /search
    Disallow: /feed
    Disallow: /comments/feed
    Disallow: /feed/$
    Disallow: /*/feed/$
    Disallow: /*/feed/rss/$
    Disallow: /*/trackback/$
    Disallow: /*/*/feed/$
    Disallow: /*/*/feed/rss/$
    Disallow: /*/*/trackback/$
    Disallow: /*/*/*/feed/$
    Disallow: /*/*/*/feed/rss/$
    Disallow: /*/*/*/trackback/$

  • The topic ‘Robots.txt File Question’ is closed to new replies.