Removing WP blog from search engine robots

  • I’m having some problems removing my blog made with Word Press from search engines. I’ve added a meta tag to the index template like so:
    <META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
    right after the <head> tag. However, all search engines still find all of my archived pages (ex: index.php?m=20040724 for a post on July 24, 2004). When I view the source code for these pages, they all seem to have the meta tag on them, so I don’t understand why search engines are still crawling my site. I’ve also tried to use a robots.txt file, but forever reason that doesn’t seem to be working either. I tried using Google’s automatic removal system for the index.php file, but that seems to have only removed the index.php file and not all of my archives. What am I doing wrong? Also, once the meta tags are in place, how long should it be taking for search engines to stop crawling my site? Thanks for the help.

Viewing 6 replies - 1 through 6 (of 6 total)

  • Have you tried a robots.txt ?
    Even so, many SE’s completely ignore any directives you want to give them and then the only route to take is through a comprehensive .htaccess file
    A VERY long thread (it actually spans 2 threads) on using .htaccess to ban SE’s is here : https://www.webmasterworld.com/forum92/413.htm
    It’s complex stuff …….

    And you can not undo indexing which has already taken place.

    Thread Starter cat

    (@cat)

    I have emailed my domain’s administrator to see what they say about the robots.txt file issue.
    Is there really no way to remove already indexed pages? My site is turning up on very random searches, and I’d prefer that all of my archives be removed if possible. I can’t manually go through and remove every page via the automatic removal system, and it seems like asking for index.php to be removed doesn’t remove things like index.php?m=20040724. If I get the robots.txt file to work, would that remove already indexed pages?
    Thanks for the help.

    indexed pages are already in the SE’s database and will be served to people looking.
    Google are fairly decent – just email them and ask that your url (or part thereof) is removed from their database. They have done this for me before.
    Other SE’s couldn’t care less in my experience ….

    inktomi actually gave me a karma point on my blog. inspite of the fact that the action file was completely banned from the robot.txt file.

    Thread Starter cat

    (@cat)

    I just came up with a plan I think will work but let me know if it won’t…
    If I just change the URL to my blog and so instead of putting it in a directory called /blog/ I put it in something like /myblog/ , and make sure that my template includes the proper META tags to disallow robots from visiting it, it _should_ get rid of all of the indexed pages in the search engine’s database after their next crawl, and not reindext the ones with the META tags… correct? I hope so, lol.

Viewing 6 replies - 1 through 6 (of 6 total)
  • The topic ‘Removing WP blog from search engine robots’ is closed to new replies.