robots.txt and AMP

  • Resolved mbeerli

    (@mbeerli)


    4 years, 5 months ago

    I use here and there a free brokenlinkchecker.
    I excluded via Disallow: /IT/ the Italian language and so on.
    Is there a way to exclude the AMP pages? There is no need to verify for a broken link checker the AMP pages.
    I tried
    Disallow: /*amp$
    Disallow: /*amp=1$
    but it does not seem to work.
    Any suggestions?

    The page I need help with: [log in to see the link]

Viewing 1 replies (of 1 total)

  • @mbeerli thanks for your inquiry.

    If you’re using robots.txt to make the exclusion, then you may find this reference from Search Console as well as this testing tool helpful for defining and checking rules. Since you’re using Reader mode, note that posts will typically be available at /amp/ (and pages ?amp) so perhaps you need an additional rule that includes the last forward slash.

    Note that some of these references/ tools are for the Googlebot specifically, so be mindful that you don’t unintentionally block your AMP URLs from that search. You may want to check directly with the brokenlinkchecker for rules specifically excluding URLs from their platform.

Viewing 1 replies (of 1 total)
  • The topic ‘robots.txt and AMP’ is closed to new replies.