Blocked by Robots.txt in Google Search Console

I am using NGINX and plugins such as W3 Total Cache

• This reply was modified 4 years, 7 months ago by sagarchaudhuri.

Hi,

Could you confirm you are using the latest version of WordPress 5.5, Yoast SEO for WordPress 14.8.1, and the rest of your plugins and themes?

Upon checking your homepage https://www.hifives.in/, we don’t see any ‘noindex’ tag or any other directive that could cause it from not showing in the search results. Upon checking the robots.txt file here it looks to be the default, and doesn’t suggest anything that would block the homepage.

If you try to request for Indexing in Google Search Console for your homepage URL, could you let us know if what the specific error or issue is by providing us with a screenshot and more details? You can follow the steps here — https://yoast.com/help/fetch-as-googlebot/

We are using WordPress 5.5 and Yoast SEO Plugin Version 14.8.1.

This is the full list of active plugins:

a3 Lazy Load
Settings | Deactivate | Troubleshoot
Speed up your site and enhance frontend user’s visual experience in PC’s, Tablets and mobile with a3 Lazy Load.
Version 2.4.1 | By a3rev Software | View details | Support

API KEY for Google Maps
Adds API KEY to Google maps calls if they have been enqueue correctly.
Version 1.2.0 | By GeoDirectory | View details

Async JavaScript
Async JavaScript gives you full control of which scripts to add a ‘async’ or ‘defer’ attribute to or to exclude to help increase the performance of your WordPress website
Version 2.20.03.01 | By Frank Goossens (futtta) | View details

Blog2Social: Social Media Auto Post & Scheduler
Auto publish, schedule & share posts on social media: Facebook, Twitter, XING, LinkedIn, Instagram, … crosspost to pages & groups
Version 6.4.2 | By Blog2Social, Adenion | View details

Duplicate Page
Duplicate Posts, Pages and Custom Posts using single click.
Version 4.3 | By mndpsingh287 | View details

Everest Forms
Drag and Drop contact form builder to easily create simple to complex forms for any purpose. Lightweight, Beautiful design, responsive and more.
Version 1.7.0.3 | By WPEverest | View details | Docs | Free support

Fast Velocity Minify
Improve your speed score on GTmetrix, Pingdom Tools and Google PageSpeed Insights by merging and minifying CSS and JavaScript files into groups, compressing HTML and other speed optimizations.
Version 2.8.9 | By Raul Peixoto | View details

Font Awesome
The official way to use Font Awesome Free or Pro icons on your site, brought to you by the Font Awesome team.
Version 4.0.0-rc20 | By Font Awesome | View details

Health Check & Troubleshooting
Checks the health of your WordPress install.
Version 1.4.4 | By The www.remarpro.com community | View details

Imagify
Dramaticaly reduce image file sizes without losing quality, make your website load faster, boost your SEO and save money on your bandwidth using Imagify, the new most advanced image optimization tool.
Version 1.9.10 | By WP Media | View details

Intergeo – Google Maps Plugin
A simple, easy and quite powerful Google Map tool to create, manage and embed custom Google Maps into your WordPress posts and pages. The plugin allows you to deeply customize look and feel of a map, add overlays like markers, rectangles, circles, polylines and polygons to your map. It could even be integraded with your Google Adsense account and show ad on your maps.
Version 2.3.2 | By Themeisle | Visit plugin site

Themify Builder
Build responsive layouts that work for desktop, tablets, and mobile using intuitive “what you see is what you get” drag & drop framework with live edits and previews.Version 4.7.4 | By Themify | View details | View Changelogs

W3 Total Cache
The highest rated and most complete WordPress performance plugin. Dramatically improve the speed and user experience of your site. Add browser, page, object and database caching as well as minify and content delivery network (CDN) to WordPress.
Version 0.14.4 | By BoldGrid | View details

WP Call Button
Deactivate | Troubleshoot | Settings
This plugin enables visitors on your website to call your business phone number by adding a call button at a prominent location on your website.Version 1.0.1 | By Syed Balkhi | View details

WP Mail SMTP
Reconfigures the wp_mail() function to use Gmail/Mailgun/SendGrid/SMTP instead of the default mail() and creates an options page to manage the settings.
Version 2.3.1 | By WPForms | View details
Enable auto-updates

WP Social Chat
Social Chat allows your visitors to contact you or your team through WhatsApp chat with a single click.
Version 4.7.0 | By QuadLayers | View details
Enable auto-updates

Yoast SEO
The first true all-in-one SEO solution for WordPress, including on-page content analysis, XML sitemaps and much more.
Version 14.8.1 | By Team Yoast | View details

The Theme is SpaciousVersion: 1.7.7
By ThemeGrill

The request for indexing on Google Search Console is giving the following error:

https://drive.google.com/file/d/1ff0K5P3ZGxCLqPXH_t4WNENmNqzsycOP/view?usp=sharing

Hi,

We can’t view the link as it informs us that we don’t have the necessary permissions. Could you please provide us with a screenshot using imgur or pasteboard that highlights the result you are getting when requesting for indexing in Google Search Console?

Please find the image link below:

https://pasteboard.co/JoKjESj.png

There are 14 URLs that are also being blocked by the robots.txt

URL	Last crawled
https://www.hifives.in/key-elements-of-a-perfect-employee-recognition-program/	23-08-2020
https://www.hifives.in/gamification-as-a-means-of-enhancing-employee-engagement/	11-08-2020
https://www.hifives.in/5-signs-of-disengaged-employees-that-organizations-must-stay-alert-to/	10-08-2020
https://www.hifives.in/can-lack-of-employee-recognition-program-hurt-an-organization/	10-08-2020
https://www.hifives.in/an-essential-guide-to-building-a-great-organizational-culture/	10-08-2020
https://www.hifives.in/the-significance-of-an-effective-employee-recognition-program-for-an-organization/	10-08-2020
https://www.hifives.in/5-tips-for-organizations-to-keep-their-work-from-home-employees-engaged/	10-08-2020
https://www.hifives.in/best-appreciation-messages-for-employee-recognition/	10-08-2020
https://www.hifives.in/useful-tips-to-maximize-the-productivity-of-employees-working-from-home/	10-08-2020
https://www.hifives.in/employee-achievements-that-organizations-must-definitely-recognize/	10-08-2020
https://www.hifives.in/checklist-for-organizations-reopening-offices-post-covid-19-lockdown/	10-08-2020
https://www.hifives.in/why-work-from-home-might-be-the-new-normal-in-the-post-covid-world/	10-08-2020
https://www.hifives.in/empathy-among-remote-co-workers-during-this-crisis-is-the-need-of-the-hour/	10-08-2020
https://www.hifives.in/everything-you-need-to-know-about-employee-pulse-surveys/	29-07-2020

• This reply was modified 4 years, 7 months ago by Yui.
• This reply was modified 4 years, 7 months ago by Yui. Reason: please use CODE button for proper formatting

@sagarchaudhuri We tried checking the image in the link you provided but nothing is loading when we view the image link (see image here).

Can you please try uploading and sharing the image link again?

@devnihil

The link works fine. I tried opening it in different browsers and in incognito mode. Here’s it again.
https://pasteboard.co/JoKjESj.png

Thanks for sharing the relevant screenshot. We have tested your site https://www.hifives.in/ on Google search and can confirm that it’s appearing in the search results as expected.

In addition, we can also confirm that there’s no disallow rules in the robots.txt that is blocking any URLs on your site. So, the warnings you are currently seeing in Google search console is a false positive that you can ignore.

In addition, we can also confirm that Yoast SEO is working as expected on your site. So, we are not sure why Google search console is showing you the false-positive warnings. You may want to reach out to Google Webmaster Support for further clarification on this.

We are going ahead and marking this issue as resolved due to inactivity. If you require any further assistance please create a new issue.