Brian M

This was just what I was looking for! Thank you!

Thanks for the response, here is an example of a pretty link that is being crawled: https://www.mymillennialguide.com/acorns

Unfortunately, I’m not using a slug prefix. Is there anyway to prevent links from getting crawled without having a slug prefix?

• This reply was modified 1 year, 10 months ago by Brian M.

Hi Tyler,

Don’t mean to hijack the thread, but I’ve already done this. They aren’t being index but still being crawled.

I’ve got hundreds of prettylinks and they are still getting crawled by Google which is taking up crawl budget. Is there a way to use robots.txt to prevent PrettyLinks from being crawled?

• This reply was modified 1 year, 10 months ago by Brian M.

I also came here to ask this, does anyone know how to block Google from crawling prettylinks by using robots.txt?