Robots.txt is not valid

@crosby87

Upon checking the robots.txt file https://dolgospenz.hu/robots.txt we don’t see any problem with the generated robots.txt file and is per the recommended standard.

Therefore, this seems to be a false positive error.

Could you run your file through Google’s robots.txt file tester and see if it is able to detect it properly? If that’s the case, it’s a false positive. You can learn more about it here –?https://support.google.com/webmasters/answer/6062598/

@suascat_wp Many thanks for your help!

Are such “false positive errors” frequent?

I found this Google support forum ticket which suggests that not specifying User-Agent can be one of the problems. Upon checking https://dolgospenz.hu/robots.txt I do not see the User-Agent being defined, though I have to say I have zero coding experience.

Could you please confirm if it is indeed alright and it is just a false error from Google’s side?

Thanks again!

• This reply was modified 1 year, 3 months ago by crosby87.

@wendihihihi thanks for your reply.

Do I just need to click on Test and that’s it? Apologies, but I do not have experience with this.

If so, I did that and two things happened: 1) the “Test” button turned to “Allowed” and the “Disallow:” line got highlighted in green. Is that it? What does this mean?

You can see a screenshot here: https://ibb.co/7rrdn5Y

Thanks again for the help!

User-agent: * means that any following instructions apply to all crawlers. I see no reason to specify all user agents by name. That will make the file clumsy.

Also, your screenshot shows the robots.txt tester found no errors in your file, so it’s fine

• This reply was modified 1 year, 3 months ago by Maybellyne.

This thread was marked resolved due to a lack of activity, but you’re always welcome to re-open the topic. Please read this post before opening a new request.