• I just got an email from Google saying:

    Googlebot cannot access CSS and JS files

    on my site.

    Seems the robots.txt file is disallowing wp-admin/ and wp-includes/
    which it has always done.

    BUT there’s CSS and/or JS in there that Google wants to be able to see.

    How to fix? Allow all?

    Has the Googlebot lost its mind?

    THANKS!

Viewing 15 replies - 46 through 60 (of 94 total)
  • The following code works for me

    #Googlebot
    User-agent: Googlebot
    Allow: *.css
    Allow: *.js
    
    # global
    User-agent: *
    Disallow: /wp-admin/
    Disallow: /wp-includes/

    Thank You “mlepisto” for your guidance.

    omurphy, you’re welcome to do as you wish.

    Your response you quoted are specific to the wp-admin pages.

    Theme files – which aren’t in the wp-admin directory – and may include items such as timthumb or other automatic image resizing can cause a significant server performance hit when crawled, especially if many of them are crawled successively as I mentioned before.

    mlepisto why to use

    Allow: /*.js*
    Allow: /*.css*

    rather than

    Allow: /*.js
    Allow: /*.css

    do any files have characters after .js or .css in their names?

    Some of mine do, for example:

    /wp-content/plugins/custom-contact-forms/css/jquery-ui.css?ver=3.3.1

    If you have Google webmaster tools, you can ask it to fetch and render your page and it’ll then tell you what the problem files are

    You can then allow them individually (and test) in webmaster tools. I allowed the files they mentioned and all ok now

    Some of mine do, for example:

    /wp-content/plugins/custom-contact-forms/css/jquery-ui.css?ver=3.3.1

    This is VERY common in wp themes.

    Lots of Wordpest and Drupal sites are getting “Can’t find your CSS or JS” warning from Google today.

    They changed their algorithm and CSS and JS files located in core locations like /misc or /sites/all/modules in Drupal, or /wp-admin/ in WordPress, are technically blocked by the default robots.txt rules for those systems.

    If you’re using CSS and JS aggregation / minification and the destination is outside of what’s blocked by robots.txt then you’d be good.

    Otherwise, a simple fix, though not necessary a long-term one, is to add explicit Allow: rules for Googlebot, as this guy describes -> https://www.beachfrontsolutions.com/how-to-fix-googlebot-cannot-access-css-and-js-files.html

    By editing Robots.txt i able to solved this issue
    I am using Hueman WordPress theme for my blog https://codepedia.info
    Set my robots.txt as
    Method 1:

    User-agent: *
    Disallow: /wp-admin/
    Disallow: /wp-includes/
    allow: /wp-content/plugins/
    allow: /wp-content/themes/

    Method 2:

    #Googlebot
    User-agent: Googlebot
    Allow: *.css
    Allow: *.js
    
    # Other bot spider
    User-agent: *
    Disallow: /wp-admin/
    Disallow: /wp-includes/

    Using Method 2 issue will be fixed only for GoogleBot. If in future Bing / Yahoo other bot also throws same error then just apply method 1
    Detailed Article : How to fix [WNC-658001] Message “Googlebot cannot access CSS and JS files”

    As Satinder has said, his method will work fine

    #Googlebot
    User-agent: Googlebot
    Allow: *.css
    Allow: *.js
    
    # Other bot spider
    User-agent: *
    Disallow: /wp-admin/
    Disallow: /wp-includes/

    Changing the robots.txt works for not blocking anything on my website but I still get warnings that other js (eg/ addthis widget) are blocked. Any ideas about this, what am I missing?

    So, no comment from WP or am I missing something? Yoast is widely cited but no comment. Can anyone provide links to responses to Google’s messages?

    Its not a google or wp bug.

    Apparently google just started sending out the JS/CSS warnings today (even though they have been using this method for some time) which would explain why we all are just finding out:
    https://www.thesempost.com/new-google-search-console-warning-for-googlebot-cannot-access-css-and-js-files/

    More info above.

    I guess its OK to allow all now. As google is smart enough apparently not to index wp core files. They started targeting css/js a while back on their last update, but are now just sending out warnings. Here are some articles:
    https://yoast.com/google-panda-robots-css-js/
    https://yoast.com/wordpress-robots-txt-example/

    Hi Guys,

    As long as you are not disallowing your plugin and theme folders you have nothing to worry about.

    I’d stop short of calling it a bug, but this was sent to potentially millions of sites today, and I have a feeling we will see either a retraction or some kind of clarification from Google soon.

    I tried to access my css and js files with fetch as google and test it from search console. It’s crawlable, So i think its just a bug.

    Google search console keep telling me
    Googlebot cannot access CSS and JS files on https://technotif.com/

    @jameshoward2122 you are blocking “wp-includes” – which is in turn blocking a js file… so technically it is accurate for you…

Viewing 15 replies - 46 through 60 (of 94 total)
  • The topic ‘Googlebot cannot access CSS and JS files’ is closed to new replies.