Search Console Errors

Hi,

No, this is not the right forum to ask these questions ??

…but…

as a fellow 5XX-error-code victim, here’s what I’ve learned about this issue:

This is fundamentally a Cloudflare-related issue, though the problem may be caused at the origin servers, either by some of your hosting company firewalls, or by the firewall in security plugins such as Wordfence.

Never mind about the many 500 error page in the aw-snap.info report, because typically when a bot tries to get too many pages off Cloudflare’s CDN servers at the same time, some sort of limit is reached, and this results in the 5XX-error codes. It may be that when Cloudflare tries to get all those pages from your origin server (your hosting provider), it takes too long for it to respond, and Cloudflare is not gonna wait. Or it may be that Cloudflare itself has a firewall rule or limit, or that some plugin such as Wordfence (if you use it) has a firewall rule that limits the access of bots to your site. If you start asking around, get ready for the blame game. What you may want to do is to whitelist any online tools and legit bots you may use, such as Googlebot, Spider Frog, aw-snap.info etc) from the firewalls at both the Cloudflare level and at the origin level (Wordfence or any other security plugin, whatever firewall your hosting provider may have etc.)

Be patient, because even after you whitelist them all, there’s still a good chance the 5XX-error pages continue to appear. Try to use these bots in off-peak hours, that helped me.

If the issue persists, you may want to contact Cloudflare support, they will help you figure out what may be wrong with your configuration.

As for your DNS, I saw nothing wrong with that report. All Cloudflare-protected websites will have a similar report. The missing MX record is missing for a reason: you don’t have one, since you use a Gmail address instead of a [email protected] address.

Your robots.txt is fine, except that I don’t see any reason for allowing/disallowing wp-admin or admin-ajax.php. You should have other protections against unauthorized access to the /wp-admin/ area, either via Cloudflare, Wordfence, or some other security tool. Someone with website security expertise may be able to help you with that, if you ask in the appropriate forums or even on Twitter.

Good day CB, thanks for your response and my apologies for posting in the wrong area. Kind of posting in any place I think might be of help as this has frustrated me to no end.

I am still having issues and having zero technical knowledge, really makes it difficult. I have reached out to Cloudflare to see if they can check my configuration in case there is something causing the issue. The have not been much help really, they just keep pointing to a general guideline.

Thank you for your suggestions, it helps me when I discuss it with other people when trying to troubleshoot.

Much appreciated.
Kevin

Will have to close this thread as this is a Non-Yoast issue. Thank you for understanding.