Robots.txt blocking Google
-
I have been told by Google News that parts of my site are being blocked from their crawlers, therefore, they can’t include my site in Google News until this is corrected. I’ve done everything I can think of to fix it, even deactiving the WordPress SEO plugin and installing Yoast SEO plugin. The latter I heard is easier to deal with.
No matter what I do, stuff seems to be blocked from the crawlers and some it seems to be Google AdSense and Javascript files. Here’s a rundown of what’s blocked.
https://pagead2.googlesyndication.com/pagead/show_ads.js Script Blocked
https://www.tqlkg.com/image-7636937-10386908 Image Blocked
https://www.lduhtrp.net/image-7636937-10675279 Image Blocked
https://scontent-a.xx.fbcdn.net/hprofile-xfa1/v/l/t1.0-1/p50x50/10805808_10154898303465416_8412066579935943526_n.jpg?oh=d6f0a147a5f2d9a853b57932bd79d20c&oe=55032BE5 Image Blocked
https://cdn.syndication.twimg.com/widgets/timelines/358011796092772352?domain=themusicuniverse.com&lang=en&t=1577185&callback=twttr.tfw.callbacks.tl_358011796092772352_1&suppress_response_codes=true Script Blocked
https://public-api.wordpress.com/wp-admin/rest-proxy/My robots.txt file looks like this:
User-agent: *
Disallow: /wp-admin/Sitemap: https://themusicuniverse.com/sitemap_index.xml
Sitemap: https://themusicuniverse.com/sitemap.xmlAny help to fix this would be greatly appreciated!
Thanks,
Buddy
- The topic ‘Robots.txt blocking Google’ is closed to new replies.