Hide development site from search engines

  • Hi,

    I have been developing a new company website with wordpress, and I see that it is already coming up in search engines. A good thing for down the road, a bad thing until it is done. Is there any way to stop this form happening? THanks!
    Matt

Viewing 3 replies - 1 through 3 (of 3 total)

  • If you want to exclude all the search engine spiders from your entire domain, you would add a robots.txt file your website folder and put the following text in it:

    User-agent: *
Disallow: /

    Use with care as the site won’t get indexed by search engines while the file is still there. See here for more information on robots.txt.

    Thread Starter TheJoyfulOrganizer

    (@thejoyfulorganizer)

    Here’s the thing, if I have installed WordPress on my hosting server that contains my current company site, will that not stop all search engine spiders and impact my current working site too? So that would probably be a bad thing overall, right?

    It should only apply to subfolders, so if wordpress is installed to https://www.yourdomain.com/wordpress/ and you put robots.txt in https://www.yourdomain.com/wordpress/robots.txt, it should be fine.

    To identify each file you want to exclude, and not folders and domains, you can also put this information in meta fields in the html or wordpress theme template directly, for example:

    <html>
<head>
<title>...</title>
<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
</head>

    More info on the meta no robots approach here.

    Another idea would be to forward all traffic coming to your under-development site to the new site, using something like this:

    <META
HTTP-EQUIV="Refresh"
CONTENT="5; URL=autoforward_target.html">
Viewing 3 replies - 1 through 3 (of 3 total)
  • The topic ‘Hide development site from search engines’ is closed to new replies.