Auto index new posts and updated posts

If WP Search With Algolia automatically indexes new posts when they are created and if existing posts get re-indexed when they are updated.

Yes, it should all be self-maintaining on new content and updated existing content.

Out of curiosity, in what way were you seeing things that seemed to indicate otherwise? For example are you using something like ACF or other custom field plugins and updating those didn’t seem to update the index?

Here’s the actions/filters we use to listen to post changes:
https://github.com/WebDevStudios/wp-search-with-algolia/blob/main/includes/watchers/class-algolia-post-changes-watcher.php#L72-L100

Most specifically the save_post action hook.

Thanks for pointing me to the Github code. Seems like you have it all accounted for. It’s not something we’ve consistently seen, it might just be a fluke. We have an existing page that was updated and it was no longer in our index. I’ll keep an eye out for any future anomalies or examples of the indexing not working as expected. I appreciate the quick reply!

Welcome.

I think I figured out what’s going on and why I thought that the auto-indexing of new posts wasn’t occurring. We’re running a multilingual site using WPML. All of our posts share the same Algolia index (i.e. they are not separated by language). We add extra WPML language metadata to each post in the Algolia index via the algolia_post_shared_attributes and algolia_searchable_post_shared_attributes hooks.

The metadata that gets added above is an array, which includes a language code and locale. We created a facet using the WPML locale via the algolia_searchable_posts_index_settings and algolia_posts_index_settings hooks so that search results on our website are only shown if the indexed record’s wpml.locale value matches the current language that they’re viewing our website in.

Where I think the issue is happening is that the indexing of the newly created post happens before the save_post tasks that WPML is doing, which means the array of custom WPML metadata being pushed with the new post to our Algolia index is empty.

I was searching in our Algolia index for newly created posts (using a filter for English pages) and nothing came up. However, when I removed that filter and searched through all pages, the new post was there, with null values for the WPML attributes.

So….after that long background information, is there a way to delay the save_post actions that WP Search With Algolia runs so that they happen after WPML assigns the appropriate language data to the new post?

Good day,

That’s all making sense to me, and I’ve also been delving into record updating around extra metadata recently, so I have a couple things to point out.

First up, we have a algolia_watch_post_meta_keys filter that you can add meta keys to watch for changes on, and re-index at that point. I suspect, based on already attempted tests and other confirmations, if you added the associated WPML meta keys to the array, things would start updating more readily. See the function at https://github.com/WebDevStudios/wp-search-with-algolia/blob/2.6.2/includes/watchers/class-algolia-post-changes-watcher.php#L229-L251

Second up is that we have a recently opened issue on GitHub regarding re-index point, with as you mentioned, the current point being save_post and that runs BEFORE meta/term information has been updated. Meanwhile, the wp_after_insert_post still runs in the same range of WP core’s update process, but at that point meta and term data has been saved. You can see that open issue over at https://github.com/WebDevStudios/wp-search-with-algolia/issues/384 and feel free to chime in as well with your thoughts.

Well that’s definitely fortuitous for me that you already have this kind of behavior on your mind. I hooked into the algolia_watch_post_meta_keys filter and had some success with a caveat.

The WPML language information that I add to each record in our Algolia index is fetched using WPML’s wpml_post_language_details hook which pulls metadata from different WPML database tables, not from the standard postmeta table (the main method that is called is here). So, I couldn’t add any new metakeys to the array of keys in the algolia_watch_post_meta_keys filter. However, there are a few WPML metakeys that exist in the postmeta table, so I used one of them (_wpml_word_count) to serve as a proxy, thinking that this metadata record would get updated around the time the other WPML language metadata is set. That ended up being true, new posts that I create get added to our Algolia index with the proper WPML language attributes.

The caveat is, when I duplicate a new post from English (our site’s default language) into another language on the site, that duplicate page gets pushed to our Algolia index without any of the language data set. When I eventually make an update to that duplicate page (e.g. translate it into the target language or change the slug or do something) then the proper language attributes get assigned to the record.

Not sure how to best handle that last caveat detail. Sounds like you’re close to an overall working situation.

I wonder if it’d all work like expected if you temporarily changed the save_post hook to wp_after_insert_post

We don’t have anything imminent planned for releases after we got 2.6.2 out earlier this week, in case you want to test that out.

Are you saying to change the save_post hook in WP Search with Algolia codebase to wp_after_insert_post as mentioned and referenced above in the Github issue?

Yes, for sake of testing all the WPML parts that you’re dealing with in your caveats.

Will do, can you confirm that it’s this save_post hook that I’d be replacing?

I can confirm that changing the post watcher to listen on wp_after_insert_post worked as desired.

Yes, and good to know.

Also to be certain, we still need to discuss the change, but that’s the one we’d be discussing.

Thanks again for the quick support. I’ll leave the fix in place on our website for now and change it back if I start noticing any weird behavior. I went ahead and made a comment in the Github issue that you referenced earlier, if that helps build your case to making the change.

Thanks Kory.