Re-Index process hangs on large number of products

Hello,

So, you see in console “Request failed: error admin.js:501:25″ and after this the re-index process is stuck? It is strange because after this error plugin must try to restart index process from the last indexed product.

Anyway I also made some changes in the plugin source code. Now when you have any problem during re-index and its just stuck than you can refresh settings page and click on ‘Reindex table’ button again. This will start re-index from the last indexed product.
Note: This works only if after index errors passed no more than 1 hour.

I doesn’t release this changes yet but you can add them manually. Please open includes/class-aws-table.php file, find function reindex_table and replace its all with updated one

/*
         * Reindex plugin table
         */
        public function reindex_table( $data = false ) {

            global $wpdb;

            $index_meta = $data ? $data : $_POST['data'];
            $status = false;

            // If something goes wrong during last index start from latest indexed product
            if ( 'start' === $index_meta ) {
                $aws_index_processed = get_transient( 'aws_index_processed' );

                if ( $aws_index_processed ) {
                    $index_meta = $aws_index_processed;
                }
            }

            // No current index going on. Let's start over
            if ( 'start' === $index_meta ) {

                $status = 'start';
                $index_meta = array(
                    'offset' => 0,
                    'start' => true,
                );

                $wpdb->query("DROP TABLE IF EXISTS {$this->table_name}");

                $this->create_table();

                $index_meta['found_posts'] = $this->get_number_of_products();

            } else if ( ! empty( $index_meta['site_stack'] ) && $index_meta['offset'] >= $index_meta['found_posts'] ) {
                $status = 'start';

                $index_meta['start'] = true;
                $index_meta['offset'] = 0;
                $index_meta['current_site'] = array_shift( $index_meta['site_stack'] );
            } else {
                $index_meta['start'] = false;
            }

            $index_meta = apply_filters( 'aws_pro_index_meta', $index_meta );
            $posts_per_page = apply_filters( 'aws_index_posts_per_page', 10 );

            $args = array(
                'posts_per_page'      => $posts_per_page,
                'fields'              => 'ids',
                'post_type'           => 'product',
                'post_status'         => 'publish',
                'offset'              => $index_meta['offset'],
                'ignore_sticky_posts' => true,
                'suppress_filters'    => true,
                'orderby'             => 'ID',
                'order'               => 'DESC',
                'lang'                => ''
            );

            $posts = get_posts( $args );

            if ( $status !== 'start' ) {

                if ( $posts && count( $posts ) > 0 ) {
                    $queued_posts = array();

                    foreach( $posts as $post_id ) {
                        $queued_posts[] = absint( $post_id );
                    }

                    $this->fill_table( $queued_posts );

                    $index_meta['offset'] = absint( $index_meta['offset'] + $posts_per_page );

                    if ( $index_meta['offset'] >= $index_meta['found_posts'] ) {
                        $index_meta['offset'] = $index_meta['found_posts'];
                    }

                    set_transient( 'aws_index_processed', $index_meta, 60*60 );

                } else {

                    // We are done (with this site)

                    $index_meta['offset'] = (int) count( $posts );

                    do_action('aws_cache_clear');

                    update_option( 'aws_pro_reindex_version', AWS_PRO_VERSION );

                    delete_transient( 'aws_index_processed' );

                }

            }

            if ( $data ) {
                return $index_meta;
            } else {
                wp_send_json_success( $index_meta );
            }

        }

One more note – by default plugin index 10 products per one iteration. It is optimal value but if you have too many products than index process can take long time to finis. You can try to change this value by using following code snippet

add_filter( 'aws_index_posts_per_page', 'my_aws_index_posts_per_page' );
function my_aws_index_posts_per_page( $number ) {
    return 50;
}

But be careful with it. You you notice any new errors after adding this – remove this code.

Regards

Thank you! I have made both modifications and I’m running right now. The change in the plugin file worked – after applying the new function code, I restarted the re-index process and it started where it had left off.

I also added the code in functions.php to process 50 products at a time during indexing. I’ll let you know how that works.

Thanks!

Those two changes made a huge difference, thank you! The process completed successfully in just a few hours.

Excellent support!

Glad to help.