Is this the Best Robots.txt Example?

  • Using Google webmaster tools I realized my robots.txt file was not optimal for crawling and indexing the blog.

    I cam across what WordPress says is a good example of a SEO robots.txt file. I was wondering if you think this is the best info for an average WordPress blog to have in its robots.txt file.

Viewing 3 replies - 1 through 3 (of 3 total)
  • Moderator Samuel Wood (Otto)

    (@otto42)

    www.remarpro.com Admin

    If you use WordPress in the root of your site, it auto-generates a robots.txt output for you, based on the privacy settings. I tend to just leave it at that.

    You can, of course, make your own based on your own site, but I think most people just want their whole site to be indexed.

    Thread Starter totalgaz

    (@totalgaz)

    Hay Otto,

    So the example WordPress gave in their codex page is it the same as it does automatically for a standard WordPress blog?

    Moderator Samuel Wood (Otto)

    (@otto42)

    www.remarpro.com Admin

    No, the example is just an example. It’s not a recommended example or something like that.

Viewing 3 replies - 1 through 3 (of 3 total)
  • The topic ‘Is this the Best Robots.txt Example?’ is closed to new replies.