Index rebuild gets stuck – no way to resume

  • We’re running version 1.1.9 of this plugin, but this problem has happened in previous versions. Our database has ~120k users in it. We’ll consistenly have the plugin showing the “Background user index building in progress: [number here]% complete” warning, and it’ll be anywhere from 5% to 95% that it gets stuck at.

    To get it unstuck, we’ve played with adding if ( $force ) $task->clearStatus(); inside of the 3 tasks in maybeIndexEverything in indexer.php, but that just masks the problem.

    We’ve moved INDEX_WP_USERS_FOR_SPEED_BATCHSIZE down (to 500) and up (to 10000), but the problem remains.

    WP_CRON is disabled and is called server-side by our provider. I wonder if PopulateMetaIndexRoles gets terminated by our provider in situations where it runs for more than 120s, and that stops the doTaskStep from setting up the next schedule?

    Questions:

    • Do you think we’re on the right track that the event is timing out, which is why the task never resumes/completes?
    • Aside from switch host or increase timeout (which they won’t allow), any other ideas to prevent this?
    • Would you consider adding plugin functionality to detect/resume stalled tasks?
    • Would you consider a PR to add CLI functionality to resume and run the tasks, outside of the scheduled event?

    (Above all else – thanks for the great work you do on your plugins! I appreciate it all very much.)

Viewing 1 replies (of 1 total)
  • Thread Starter ecaron

    (@ecaron)

    @olliejones, in case some more data helps, I added some metrics to track the speed. At peek I’m able process about 5500 users a second, and other times its about 100 every second. All the processing time happens in the doChunk of PopulateMetaIndexRoles, and the latest run took 4 hours (non-shared environment in GCP of 509999 entries).

    All other plugins have been disabled. My assumption is that the plugin behaves best if the “Rebuild indexes” occurs at least daily. If it is something like “It only needs to work once”, then let me know and I’ll stop researching where the timeout happens.

Viewing 1 replies (of 1 total)
  • You must be logged in to reply to this topic.