Guiding SearchBots

maybe u can use ur robot file to block the index file from the spiders.
but i am not sure if the permalinks will fool the spider and let them read the index file with parameters. not sure though.
i would be interested to know if custom metatags for the home page and other pages is possible ??

Check out the info from the horse’s mouth and learn about what Googlebot does and does not do. There are plenty of site devoted to helping you with your robot.txt file as well.

Thanks for all the pointers. I did check the google page, and read up on robots.txt and all that.
<b>tcervo</b> — Could you please elaborate on what “$p” means or does, I am sorry for being so ignorant about php. I was thinking along the same lines, — of modifying the index.php with an if – else statement, but had no idea as to what to do. So yes, please let me know what the $p does/stands for.
Thanks.

The $p variable will contain the post_id when the current view is narrowed to a specific post. So the code above would tell robots to index individual post views, but not the main index page where there are multiple posts being shown.

$p is a variable that holds the post id. So, when you go to an individual post page, $p should have the post id for that post. If it doesn’t exist, then you’re most likely not on an individual post page.
I’m not sure how this works when using mod_rewrite (since I’m not using it), but I know it’s passed as a URL parameter otherwise. So, you click on a permalink it should have $p={post_id} in the URl after the index.php….
-Tony

Thanks tcervo & dougal.
I think mod_rewrite should not be a problem, since apache would rewrite the request URL to the “normal” url with ?p , ?cat etc in it before it forwards the request to wordpress’ engine, if I may call it that.
I was about to ask you, “what about ?cat= , ?m= etc…?” which was a question I had earlier too, but I guess I found my answer, since I dont wont anything but individual pages to be returned as google search results.
Just curious though, can you you do a
if($p || $cat || $m)
{where || stands for “OR”} ?
how do you say if(anything-at-all) ? , that is, if there is any parameter at all, then say this in PHP?

Just curious though, can you you do a
if($p || $cat || $m)
{where || stands for “OR”} ?

Yes, absolutely…I use or (||) frequently.

I finally got it going, and it works just as expected, which I found out by viewing the source of the different pages (the indexpage, an individual entry page etc)
Thanks tcervo!! I was getting a php parse error, which I fixed by using some of the other code in index.php as an example and guide ??
Also for the index.php page, we want the bot to follow the links,
So finally the code is …
<?php if ($p) {?>
<meta name=”robots” content=”index,follow” />
<?php
}
else
{?>
<meta name=”robots” content=”noindex,follow” /><?php
}
?>
Thanks y’all ??

I tried it too and here it works too! (I’m using mod_rewrite).
Thanks for this neat piece of code!! ??

Hi Sushubh,
Refer to https://www.google.com/bot.html#noindextags
So we have noindex, follow for the index page which means it wont be indexed but the links on it will still be followed.
I hope that explains it.

okie dokie! ??

Might want to mention that upgrading your search engine profile may involve a whole lot more than adding META. I have WP running at home and am getting in shape to publish. I plan on making the max use of the H1 tag among other things to keep those bots coming.

<meta name=”robots” content=”<?php echo (!isset($p)?’no’:”); ?>index,follow” />

feck… remove the isset, I’m so used to needing it.
<meta name=”robots” content=”<?php echo (!$p?’no’:”); ?>index,follow” />

It gets better and better!! ??