Googlebot cannot access CSS and JS files

The following code works for me

#Googlebot
User-agent: Googlebot
Allow: *.css
Allow: *.js

# global
User-agent: *
Disallow: /wp-admin/
Disallow: /wp-includes/

Thank You “mlepisto” for your guidance.

omurphy, you’re welcome to do as you wish.

Your response you quoted are specific to the wp-admin pages.

Theme files – which aren’t in the wp-admin directory – and may include items such as timthumb or other automatic image resizing can cause a significant server performance hit when crawled, especially if many of them are crawled successively as I mentioned before.

mlepisto why to use

Allow: /*.js*
Allow: /*.css*

rather than

Allow: /*.js
Allow: /*.css

do any files have characters after .js or .css in their names?

Some of mine do, for example:

/wp-content/plugins/custom-contact-forms/css/jquery-ui.css?ver=3.3.1

If you have Google webmaster tools, you can ask it to fetch and render your page and it’ll then tell you what the problem files are

You can then allow them individually (and test) in webmaster tools. I allowed the files they mentioned and all ok now

Some of mine do, for example:

/wp-content/plugins/custom-contact-forms/css/jquery-ui.css?ver=3.3.1

This is VERY common in wp themes.

Lots of Wordpest and Drupal sites are getting “Can’t find your CSS or JS” warning from Google today.

They changed their algorithm and CSS and JS files located in core locations like /misc or /sites/all/modules in Drupal, or /wp-admin/ in WordPress, are technically blocked by the default robots.txt rules for those systems.

If you’re using CSS and JS aggregation / minification and the destination is outside of what’s blocked by robots.txt then you’d be good.

Otherwise, a simple fix, though not necessary a long-term one, is to add explicit Allow: rules for Googlebot, as this guy describes -> https://www.beachfrontsolutions.com/how-to-fix-googlebot-cannot-access-css-and-js-files.html

By editing Robots.txt i able to solved this issue
I am using Hueman WordPress theme for my blog https://codepedia.info
Set my robots.txt as
Method 1:

User-agent: *
Disallow: /wp-admin/
Disallow: /wp-includes/
allow: /wp-content/plugins/
allow: /wp-content/themes/

Method 2:

#Googlebot
User-agent: Googlebot
Allow: *.css
Allow: *.js

# Other bot spider
User-agent: *
Disallow: /wp-admin/
Disallow: /wp-includes/

Using Method 2 issue will be fixed only for GoogleBot. If in future Bing / Yahoo other bot also throws same error then just apply method 1
Detailed Article : How to fix [WNC-658001] Message “Googlebot cannot access CSS and JS files”

As Satinder has said, his method will work fine

#Googlebot
User-agent: Googlebot
Allow: *.css
Allow: *.js

# Other bot spider
User-agent: *
Disallow: /wp-admin/
Disallow: /wp-includes/

Changing the robots.txt works for not blocking anything on my website but I still get warnings that other js (eg/ addthis widget) are blocked. Any ideas about this, what am I missing?

So, no comment from WP or am I missing something? Yoast is widely cited but no comment. Can anyone provide links to responses to Google’s messages?

Its not a google or wp bug.

Apparently google just started sending out the JS/CSS warnings today (even though they have been using this method for some time) which would explain why we all are just finding out:
https://www.thesempost.com/new-google-search-console-warning-for-googlebot-cannot-access-css-and-js-files/

More info above.

I guess its OK to allow all now. As google is smart enough apparently not to index wp core files. They started targeting css/js a while back on their last update, but are now just sending out warnings. Here are some articles:
https://yoast.com/google-panda-robots-css-js/
https://yoast.com/wordpress-robots-txt-example/

Hi Guys,

As long as you are not disallowing your plugin and theme folders you have nothing to worry about.

I’d stop short of calling it a bug, but this was sent to potentially millions of sites today, and I have a feeling we will see either a retraction or some kind of clarification from Google soon.

I tried to access my css and js files with fetch as google and test it from search console. It’s crawlable, So i think its just a bug.

Google search console keep telling me
Googlebot cannot access CSS and JS files on https://technotif.com/

@jameshoward2122 you are blocking “wp-includes” – which is in turn blocking a js file… so technically it is accurate for you…