Shouldn’t the Pages List show that a page is now set to noindex, nofollow

  • Resolved 22carol

    (@22carol)


    5 years, 4 months ago

    They still show Index, Follow in the list.

    The heading for that column is:
    Robots Meta

    Shouldn’t I be seeing that change in the individual Page setting right away, to confirm that the change is made? (using the Disallow button on the individual Page edit screen)

    Same as I see when I’ve made a change to Tags or Categories, and the change shows right away on the Page List?

    I tried to look at my robots and sitemap files and all attempts gave a “page not found” error.

    I’m stymied!

    The page I need help with: [log in to see the link]

Viewing 3 replies - 1 through 3 (of 3 total)
  • Thread Starter 22carol

    (@22carol)

    PS: I don’t have Yoast, deleted it when I saw they will soon be not working on non-Gutenberg sites. I will not be “upgrading” to it ever.

    Thread Starter 22carol

    (@22carol)

    Found https://carolresearch.com/researchblog/sitemap-1.xml
    BUT it has the Pages that I marked noindex, nofollow

    That one was created by JetPack.

    Smart Crawl and others say they cannot FIND a sitemap at all!
    What is going on???

    How do I get the setting on the individual Pages to work?

    • This reply was modified 5 years, 4 months ago by 22carol.
    Plugin Author Pagup

    (@pagup)

    Hi @22carol,

    Thank you for contacting our support.

    About the Post META Box feature available on each page, it’s adding instructions inside your robots.txt file, asking to “Disallow” and “Noindex” a specific page. Once selected (and saved), go check your robots.txt (through Better Robots.txt setting page), it usually appears at the very end of it. This kind of feature is very useful in case of “thank you” pages, calendar pages or any other landing page used for promotion (you don’t want these pages to be visible on SERPS).

    About this feature, you have to know that it may takes several days (or even weeks) for search engines and SEO crawlers to “take these instructions into consideration”, especially when these pages were already indexed before. In this case (already indexed), you may keep seeing them for a while on SERPs (if these pages are linked with others, through internal links, it may take even longer).

    Please note that Robots.txt and XML sitemaps are 2 very different things and are not related to each other. You can “Disallow” a page in your robots.txt file and, in the same time, have it listed in your XML Sitemap… It does not make “sense” per se but it’s not necessarily a problem. XML sitemaps (compared to HTML sitemaps) are not used by search engines to either crawl or index content on your website. They are mainly used to provide information (thanks to META DATA) about “what is the latest content available on your website” requiring to be crawled in priority by search engines. That’s it !

    If you really want to make sure this page gets not indexed by search engines, we would recommend to add a NOindex META on top of it… (which will make it disappear from your sitemaps).

    Did you notice that your robots.txt is currently not working on your website? … Make sure to use our multisite feature to create instructions specifically related to your WP directory /researchblog/.

    Hope this helps.

    Regards

    PS: if you need more help about this, please contact our support directly.

    PS2: About Yoast not being compatible (soon) with non-Gutenberg websites, we have never heard of this but we doubt that it could happen any time soon. Millions of WordPress sites are currently not updated to WP 5.2 (or at leat 5.0) & millions of others are not even using Gutenberg editor (disabled, classic editor, …)…

Viewing 3 replies - 1 through 3 (of 3 total)
  • The topic ‘Shouldn’t the Pages List show that a page is now set to noindex, nofollow’ is closed to new replies.

Tags