Unable to resume indexing

What do you read in the operation tab’s “xxx document(s) remain to be indexed” ?

It shows 0 documents remain to be indexed.

Did you re-index all documents, by first clicking on button “Empty the Solr index” ?

NO

I need to know how many posts are detected by WPSOLR, right before any indexing occurred.

WPSOLR was detected all 70 Lac posts earlier and correctly giving count of pending posts. Now it is not.

I have tried creating a new core and setting that in WPSOLR, but there also, it shows 0 posts pending indexing. I think your plugin is not checking documents per core.

The current version of WPSOLR stores the date of the last post indexed.
But previous versions stored the id of the last post indexed.

So, if you upgraded the plugin, you need to reset the information stored, by clicking on the button “Empty the Solr index”.

This is a very sad answer.
How can you change basic idea of how the plugin works?

I have 40 Lac posts indexed and cannot delete all this indexed data as it has taken weeks!!!!

This is a very bad way to upgrade any plugin, leave alone upgrade such a critical one. Also, did you specify this change in any of your change logs? I am very disappointed with this answer.

Please suggest some other way as I will loose weeks of work!!!!

Even Solr requires reloading the core for some critical upgrades. This is an issue when a great # of documents need to be re-indexed, which unfortunately happened to you.

Most WP users have a few hundreds/thousands of posts. Your millions of documents are really out of the standard scale for a WP database. Even if you could have completed your 7 millions docs indexing, you’ll anyway have the exact same problem reindexing in the future.

I have no miracle solution right now for you, but I suggest a direct Solr import from database, like https://cwiki.apache.org/confluence/display/solr/Uploading+Structured+Data+Store+Data+with+the+Data+Import+Handler

Sorry, but I am not convinced with your answer.
You should have provided some disclaimer as I am now stuck with this plugin.

I’ll update the next change logs with your remarks.

Let see what we can do to unstuck you in the meantime.

First, I’d like to confirm that the plug-in actually sees 7M docs.

Are you ready to do some investigations in your own install ?

We need first to find out what is the last post you indexed, then to store it’s ‘post_modified’ value as the last post date indexed in the plug-in.

1) Ensure your WPSOLR version is the last one

2) To get the last post indexed just query your index with:
/solr/wpsolr/select?q=*%3A*&sort=id+desc&wt=json&indent=true

Then copy the ‘displaymodified’ field content, like ‘2015-03-09 11:48:00’

3) Replace the line in class-wp-solr.php:
$lastPostDate = wp_Solr::get_hosting_option( 'solr_last_post_date_indexed', '1000-01-01 00:00:00' );

by the line (switch my example with your copied displaymodified content):
$lastPostDate = '2015-03-09 11:48:00';

4) Go to the operation tab, change your batch size with 1, synchronize your index, and stop it as soon it indexed at least one document.
At this point, the plug-in should have been reseted to the last indexed document date.

5) Switch back the line in class-wp-solr.php to it’s original value:
$lastPostDate = wp_Solr::get_hosting_option( 'solr_last_post_date_indexed', '1000-01-01 00:00:00' );

6) Reload the Operations tab page

Tell me what you can read now.

Thanks a lot for your help and suggestions.

I followed the steps outlined and here is what I got –
A total of 3903813 documents are currently in your index.
734425 document(s) remain to be indexed.

But the pending documents value does not match the remaining posts.
It should be close to 33 Lac and not just around 7 Lac!

Please check.

You can read the code that generates the query in posts. Eventually, echo $query just before line:
return $ids_array[0]['TOTAL'];

You’ll have to understand what kind of posts are in your table, and are not detected by the query.

The query is correct and showing rightly the post types selected for indexing.

SELECT count(ID) as TOTAL FROM wp_posts WHERE post_modified > %s AND ( post_status=’publish’ AND ( post_type=’company’ OR post_type=’person’ OR post_type=’location’ ) )

Yet the count is incorrect of pending documents.