Smart Phone Blocked resource

  • Resolved ronwisely

    (@ronwisely)


    5 years, 9 months ago

    Respected team ,
    Sorry to disturb you again . But i need to ask you one small question . Hope team won’t mind in giving reply for this small question
    This is the screenshot of https://prnt.sc/mdkw6g webmaster blocked link .
    When trying to fetch any /go/ link https://www.gomahamaya.com/go/yoast on redirectcheck.com/index.php then it shows what is there in that blocked link . But according to robots.txt file i have blocked all bots . Still they are able to fetch data .
    But when i try other website data with link /go/ they show bots are not allowed to fetch data .
    So my question is am i doing any thing wrong in cloaking links ? do i have to add something more on my website ? Do i have to add any special header function to my website ?
    While searching on google i found this code
    $url = “//{$_SERVER[‘HTTP_HOST’]}{$_SERVER[“REQUEST_URI”]}”;
    if (preg_match(“#/go/#”, $url))
    {
    header( “X-Robots-Tag: noindex, nofollow”, true );
    header( “Cache-Control: private”, true );
    }
    Should i add above code to my website Or any special header code i have to add via .htacess file . Yoast always recommend to add header code via .htacess instead of function.php .
    So i will request you to help me on this topic . Any help will be much appreciated .
    Regards,
    Rahul

    The page I need help with: [log in to see the link]

Viewing 4 replies - 1 through 4 (of 4 total)

  • Hi,

    We understand that Google and other search engines are finding links on your site that you do not want them to crawl. There are two ways to stop Search Engine Robots from crawling certain pages.

    1. Edit your robots.txt to provide a disallow rule. This guide explains more about how to edit the file: https://kb.yoast.com/kb/how-to-edit-robots-txt-through-yoast-seo/.

    We also have a link that explains more about how to make disallow rules: https://searchengineland.com/a-deeper-look-at-robotstxt-17573

    We also have a guide on how to cloak links that may be helpful here: https://yoast.com/cloak-affiliate-links/.

    2. Add a rule into your server that noindex certain pages. We are not sure if that code should be added into your server but you are welcomed to add it and see if it resolves the issue. We do recommend taking a backup (your host provider can help with this) of your site prior to editing the server/.htaccess files.

    Thread Starter ronwisely

    (@ronwisely)

    Respected Pcosta88 (@pcosta88) ,
    This is my website robots.txt https://www.gomahamaya.com/robots.txt . i followed everything which you mentioned on the above files . Even i tried following yoast affiliate link cloaking rule https://yoast.com/cloak-affiliate-links/ But still i am getting error my website https://snag.gy/HoOguw.jpg . Even i am able to fetch data which is inside /go/ folder using this tool https://redirectcheck.com/index.php . When i check other website /go/ link then the redirect check tool failed to crawl .
    I tried almost everything to fix my error now i am tired that’s why i am asking help from SEO masters . I just need a small guidance .
    Thanks

    Hi,

    We are not sure why when you have entered rules in your robots txt here: https://www.gomahamaya.com/robots.txt to disallow certain folders like /go/ that some tools are still able to access the file.

    We suggest contacting the https://redirectcheck.com/index.php tool authors for more information as to why they can access the file when you have blocked bots from accessing it.

    In regards to Google here: https://snag.gy/HoOguw.jpg, what is shown is expected. Google is finding links but recognizing that it has been blocked from indexing or crawling them. It then let you know what it found so you can make a decision.

    If you do not want Google to crawl those /go/ folders then everything is working as expected as Google is letting you know it found some URLs and that it is blocked from crawling and indexing them.

    Plugin Support amboutwe

    (@amboutwe)

    This thread has been marked as resolved due to lack of activity.

    You’re always welcome to re-open this topic. Please read this post before opening a new request.

    Thanks for understanding!

Viewing 4 replies - 1 through 4 (of 4 total)
  • The topic ‘Smart Phone Blocked resource’ is closed to new replies.