how to add robots.text file

Viewing 3 replies - 1 through 3 (of 3 total)

  • Try using Google. This isn’t a WordPress specific issue.

    Thread Starter tradtips

    (@tradtips)

    ok thanks.

    You need to make a robots.txt file. You can start with this:

    User-agent: *
    Disallow: /cgi-bin
    Disallow: /wp-admin
    Disallow: /wp-includes
    Disallow: /wp-content/plugins
    Disallow: /wp-content/cache
    Disallow: /wp-content/themes
    Disallow: /trackback
    Disallow: /comments
    Disallow: /category/*/*
    Disallow: /tag/
    Disallow: */trackback
    Disallow: */comments

    …and then upload it to root folder of your website (where wp-admin, wp-content, and wp-includes folders are). After you upload it you can enter https://your_domain_name.com/robots.txt to see it in a browser or go to something like: https://www.spidertest.com/ to see if it works or not etc (I did not check this one, but I know it is not able to spider my site as I am not letting anybody in right now)

Viewing 3 replies - 1 through 3 (of 3 total)
  • The topic ‘how to add robots.text file’ is closed to new replies.