about robots file and managing bots
-
I am having all my links clicked by bots and even selecting Under Attack mode in Cloudflare doesnt control it. So I found a plug that has a high rating and supposed to help with this. Blackhole for Bad Bots, they require to modify the robots.txt file, but since SEO Framework is managing this file, I am afriad to do anything with it, so asking for your advice.
my site is called buyandsellgoldbullion.com. When I look at content of robots file in the blogs filedirectory i see this content:
Begin Attracta SEO Tools Sitemap. Do not remove
sitemap: https://cdn.attracta.com/sitemap/2415510.xml.gz End Attracta SEO Tools Sitemap. Do not remove Begin Attracta SEO Tools Sitemap. Do not remove
sitemap: https://cdn.attracta.com/sitemap/2415510.xml.gz End Attracta SEO Tools Sitemap. Do not remove
inside settings of seo framework, when i click under sitemap settings, when i click view basic sitemap, this is what I see now.
https://buyandsellgoldbullion.com/robots.txt
User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.phpSitemap: https://buyandsellgoldbullion.com/sitemap.xml
https://buyandsellgoldbullion.com/robots.txt
question is whether using this plugin can cause any kind of conflict working with your plugin and is my robots file functioning properly. this site seems unaffected, but im not sure about seo.
And if you have any suggestions about blocking bots, the pls share.
hopefully, there is no problem, i just thought I would run this buy you, since your plugin manages robots file and this plugin requires the modification. so long as no problem, great. i hope it does what it says to block bots. the plugin has a 5 star rating.
Thanks for your feedback and insight.
Robert
The page I need help with: [log in to see the link]
- You must be logged in to reply to this topic.