Robots.txt from pages

what does your .htaccess look like right now?

And, honestly, why do you want to do this, just to allow ppl to look at it?

If you post you .htaccess file some experts here might be able to help you.

Personally, I wouldn’t want the average joe looking at my robots.txt file. There are things in there that a private (ie, secret URLs) that are private for a reason.

So, going back to whooami’s question, why do you want to do this?

jkozuch,

robots.txt by their very nature need to be readable. thats how they work. And guess what, if Googlebot can read it, I can too, and so can everyone else.

robots.txt are NOT intended to hide/keep private/etc.. directories, it’s intended to keep a directory from being indexed. and thats NOT the same thing. Additionally, since a robots.txt file needs to be readable, by design, putting the name of a directory that you want to be “hidden” (and I use that term loosely) inside it, defeats the purpose.

—

In a nutshell:

a directory which you desire to be “hidden” (and I use that term loosely) should not be linked to. NOT just because a searchbot will find it, but because a human will, inevitably.

a directory that you dont want indexed is probably going to be one with dynamic content, files that are downloadable, perhaps a directory that once contained ‘things’ but no longer does, and google (for instance) keeps going back to check on from time to time.

—

apples and oranges.