Failed: Robots.txt unreachable

Hi @cheekymate06,

Firstly, it’s important to note that WordPress automatically creates the robots.txt file. This file is not generated by All in One SEO. AIOSEO offers a feature that allows you to add rules to the robots.txt file for further customization.

I checked the robots.txt file of your site and it looks absolutely fine.

I’d recommend checking the status of your robots.txt file again and requesting a recrawl if necessary.

1. Log into Google Search Console: Access Google Search Console and sign in with your account.
2. Select Your Property: Choose the correct website property from the dropdown menu in Google Search Console.
3. Access Settings: On the left-hand side panel, click on ‘Settings’.
4. Check Robots.txt Status: In the Settings section, look under the ‘Crawling’ header to find the status of your robots.txt file. If it’s marked as “Valid”, this indicates that the issue has been resolved.
5. Request a Recrawl if Needed: If the status is not “valid,” click on ‘Open Report’ under the robots.txt file section. Next to your robots.txt URL, click on the three dots and select ‘Request a recrawl’. This will prompt Google to recheck your robots.txt file.
6. Wait for the Update: After requesting a recrawl, wait for Google to update the status. The ‘Checked On’ column will display the date when Google last crawled your robots.txt file.

If after these steps the issue persists or if you have any further questions, please don’t hesitate to reach out.

Hi @cheekymate06,

We haven’t heard back from you in a couple of days. I’m going to go ahead and close this thread for now. But if you’d like us to assist, please feel welcome to continue the conversation.

Thanks!