Not all pages in Google Index

Hi, do you have an account at Google Search Console (Webmaster Tools) to monitor your site in the Google index? It might tell you about problems/errors found on pages that block them from being indexed.

Or you might just have to wait a bit longer. You cannot force Google to index everything at once…

Hi! Thank you for your reply. I have Google webmasters account and I do not see mistakes for that pages. Many of them old pages. I submitted sitemap to Google but Google do not index every page even old. Maybe exist any other issue.

I suggest using the tool “Fetch as Google” (does that still exist in the new Search Console interface?) and enter one of the non-indexed pages to make sure Google gets a meaningful response.

Then modify a few of them (so the lastmod date will change) and the go to Settings > XML Sitemap and hit the Ping Search Engines button. Then wait a while and check Search Console again to see if they got indexed…

Thank you very much. I use webmasters.google.com – URL inspection and then request indexing. After XML Sitemap – “ping search engines”. But the main issue I can not add to index all pages due I do not know which not in the index. I do not understand why the sitemap does not work and not all pages in the Google index.

But the main issue I can not add to index all pages due I do not know which not in the index.

Are you saying you wish to ONLY put pages/posts in the sitemap that have not yet been indexed? That’s not a good idea. Just leave ALL pages and posts in the sitemap. The option to exclude posts/pages is only there for cases where you are using an SEO plugin to add the robots:noindex tag on certain pages, or maybe only for pages that really have no value for search engines like cookie/legal notice pages (note that without the noindex tag they still might end up being indexed).

Otherwise, it’s best to have them all in the sitemap.

This is useful for Google (each page/post has also the “last modified” info which tells search engines to come back and index the changes) but for you too: you can then compare in the Search Console (webmaster tools account) how many of your pages/posts are indexed and how many are still waiting…

I do not understand why the sitemap does not work and not all pages in the Google index.

But you cannot force Google or any other search engine to come and index pages that are not indexed yet. You can only continue adding great content and build a better site, worth the extra attention from search engines.

Thank you very much for your support. I see that sitemap with thousands pages added to Google but some or many not added to Index. I cannot say which pages exactly.
Maybe my hosting block Google robots. One day my hosting said they block Semrush SEO robots.
Last month I found big issues with hosting companies on cybersecuirity issue. Even this hostings which recommends by WordPress. No one of them can not say which cybersecuirity software they can offer to clients. And also issue with hosting plan on traffic options.
So I think hosting companies block robots.

There are many hosts that limit simultaneous visitor traffic and database requests. In fact, there is a limit to every hosting no matter the provider. The question is whether your site reaches that limit often or not. If it is often, then yes, it might impact your indexing because Google bot does not like being refused a connexion (which happens after the provider limit is reached).

If the URL inspection tool in your Search Console does not warn you about being blocked from access, then it’s probably not a systematic thing but maybe due to too large traffic volume for your hosting. Maybe your provider can tell you more about your site’s resource usage on a daily basis and if you often reach http or database request limits?