[Plugin: Relevanssi – A Better Search] Index keeps deleting itself and showing no results

Somebody else was complaining about the same problem. Unfortunately I don’t know what might be causing this. Which version of Relevanssi are you using?

I’m using Version 2.9.12. Let me know what other information might be helpful?

I am having the same issue. Also version 2.9.12. Caching is disabled. Indexing 3-4 custom post types. I can provide more details if you let me know what would be useful.

When that happens, what’s left in the index? What’s in your Relevanssi database table?

msaari, how do I check my Relevanssi database table?

Or (more accurately), are there particular things you want me to look for? Right now the index is working right, so when I go to the wp_relevanssi table I see what appears to be all the important words from my articles. I’ll bop over next time my index goes to 1 and see if looks different?

Okay. Did some sleuthing. This morning my index said “1” so I went back to wp_relevanssi. It appears to only include the most recently created post (in this case with ID 1007). Which matches what the Relevanssi options page has for “Highest Post Indexed,” incidentally.

(This is pretty typical when the index goes missing — only the most recent post/page/whatever can be searched until I rebuild the index.)

So somehow creating a new post doesn’t ADD things to the table, it REPLACES the whole table.

I hit “Build Index” and looked back at my table and it’s all full again, but I reckon as soon as we create a new post it’ll be wiped except for that post’s searchable terms.

Let me know if I can provide you with any other information.

Thanks, that was the information I was after. Unfortunately I don’t really have a clue why it would do that, it shouldn’t remove anything when adding a new document. Please let me know if that happens every time you add a new post.

Just got back to work on this site. Index was empty so I rebuilt: 333 documents in the index, highest post ID 7958. Viewing the table returns expected 18,682 rows. The results look pretty straightforward and accurate.

Add a new post. Documents in index: 1, highest post ID: 7979 (the ID of the new post). View the table: 1670 rows! all the document IDs are 7979, but I noticed a pattern:

1, 7979, 'uncategorized', 1, 4
2, 7979, 'ciarlill', 1, 5
3, 7979, 'ciarlill', 1, 1
4, 7979, 'test', 1, 1
5, 7979, 'test', 1, 0

This was repeated over and over. The post title was “ciarlill test” and the body was just “test” and it was categorized in “uncategorized”. Not sure why those 5 rows would repeat 334 times.

Rebuild the index again –
Documents in the index: 334 (coincidence??)
Highest post ID indexed: 7979

Viewing the table I get 18,687 rows. (+5 from the last index build) Just to check:

SELECT * FROM<code>www2-wordpress</code>.<code>wp_relevanssi</code> WHERE doc = 7979
14618, 7979, 'uncategorized', 1, 4
14619, 7979, 'ciarlill', 1, 5
14620, 7979, 'ciarlill', 1, 1
14621, 7979, 'test', 1, 1
14622, 7979, 'test', 1, 0

5 rows as expected, not 1670. So I think it is safe to say that something very strange is happening when new content is added. Let me know what other tests I can do or info I can provide to help.

To echo Alex, it does appear to happen every time a new post is created.

Alex, one thing you should try is to see what gets deleted when a post is added. Find the function relevanssi_remove_doc() and check out what happens with the MySQL query there. It should only delete the post in question, but it seems to delete everything for some reason.

Also, you might want to check how many times relevanssi_index_doc() is called when you create a post, drop a mail() call in there or something.

Okay some more info.

relevanssi_remove_doc query looks perfectly fine. It is getting the correct ID and being called at post creation on publish. The full creation of a post generates like 2-3 or so calls to this. Don’t think the problem is here.

relevanssi_index_doc is doing something crazy. I tried dropping a mail call in 2 places:

Beginning of functon

2154 function relevanssi_index_doc($indexpost, $remove_first = false, $custom_fields = false, $bypassglobalpost = false) {
2155     mail('[email protected]', 'RELEV INDEX CALLED', '');
2156     global $wpdb, $relevanssi_table, $post;
2157     $post_was_null = false;
2158     $previous_post = NULL;

Also after getting the post

2190     // Finally fetch the post again by ID. Complicated, yes, but unless we do this, we might end
2191     // up indexing the post before the updates come in.
2192     $post = get_post($ID);
2193     mail('[email protected]', 'RELEV INDEX CALLED', $post->ID);

Both times this seems to get called HUNDREDS of times. I had to restart Apache and I’m still clearing out my inbox and getting delayed mail. I’m afraid to approach debugging it with this method again but will try some other steps. When I would click ‘Add New’ the page would load but was still busy and would remain so until I closed the tab. Somehow adding the mail() call caused a side effect. I’m not sure if this side effect caused the 100’s of calls or if the 100’s of calls to this function all processing my mail() lead to the page hanging. Like I said I am going to continue to investigate but this is a pretty complex function and a large functional piece of relavanssi itself.

I cannot get an exact count of how many calls yet, but I have a hunch it might match the number of items previously in the index. In my case 333.

Any other ideas of where I could investigate?

Okay I added error_logging to the first line of relevanssi_index_doc.

error_log('relevanssi_index_doc called');

Add New Post. Page did not hang (assuming because it is not processing a huge number of mail() calls).

ciarlill@staff:relevanssi$ grep 'relevanssi_index_doc called' ../../debug.log | wc
    333    1332   16983

So it appears that when addind a new post relevanssi_index_doc is being called for every post that *was* in the index. In my case 333 times.

EDIT: Also added a debug message to relevanssi_build_index. Looks like this is being called on ‘Add New Post’ also.

Ouch. That’s certainly the cause of your problems.

Now the only question is why on earth relevanssi_build_index() is being called – it’s not hooked anywhere and Relevanssi only calls the function when user requests the index being built…

Here’s something you could try. Find these lines in the plugin (around line 2550):

if (isset($_REQUEST[‘index’])) {
if (isset($_REQUEST[‘index_extend’])) {

Change ‘index’ and ‘index_extend’ to something else, anything. That’ll break Relevanssi indexing feature for a while (so make sure you have a good index before you do that), then try adding a new post.

I suppose there’s a chance that for some reason something in your WP installation sets $_REQUEST[‘index’] for some reason (you might want to add some check to see if that’s actually the case), and if that’s the case, that might trigger Relevanssi indexing accidentally.

If it’s that simple, then it’s easy to fix (I’ll just change those request ids to something unique).

Thanks a lot, it’s great to have somebody doing something useful to sort this out, it’s always frustrating when several people have a serious bug that I just can’t reproduce myself.

Okay first thing I did when I got in this morning was basically as you suggested. Instead of changing the REQUEST keys to something else though I simply commented out the lines below and did some error_log output to see what that values actually were. Surprisingly NOTHING showed up. I had the same thought yesterday and briefly looked through net requests in Firebug and didn’t see anything but did not have time to do the above. So now I’m even more perplexed. Then I figure it has to be one of my personal plugins. Disable them all, still no joy. THEN I said screw it and grepped my entire WP directory for references to relevanssi_build_index… and what did I find?

relevanssi/readme.txt: if (!wp_next_scheduled('relevanssi_build_index')) {
relevanssi/readme.txt:  wp_schedule_event( time(), 'daily', 'relevanssi_build_index' );
relevanssi/relevanssi.php:function relevanssi_build_index($extend = false) {
relevanssi/relevanssi.php:            relevanssi_build_index();
relevanssi/relevanssi.php:                      relevanssi_build_index(true);
role-scoper/admin/relevanssi-helper-admin_rs.php:               if ( function_exists( 'relevanssi_build_index' ) )
role-scoper/admin/relevanssi-helper-admin_rs.php:               relevanssi_build_index();

ROLE SCOPER!

Here is the offending file:

1 <?php
  2 class Relevanssi_Admin_Helper_RS {
  3     function rvi_reindex() {
  4         if ( function_exists( 'relevanssi_truncate_cache' ) )
  5             relevanssi_truncate_cache( true );
  6
  7         if ( function_exists( 'relevanssi_build_index' ) )
  8             add_action( 'shutdown', array( 'Relevanssi_Admin_Helper_RS', 'do_reindex' ) );
  9     }
 10
 11     function do_reindex() {
 12         ob_start();  // as of version 2.9.12, no way to rebuild index w/o outputting message
 13         relevanssi_build_index();
 14         ob_end_clean();
 15     }
 16 }
 17 ?>

So I have not yet figured out WHAT this is or WHY it is or how to disable it without hacking the code. But I just wanted to share my findings as soon as possible. I can’t believe I did not think to do this before. Should have been one of the first things I did, but it appears to be a completely un-documented piece of Role Scoper. I’m sure it is there for good reason and maybe I just missed it in the FAQ or Docs for role scoper (will go back and check shortly).

Huzzah!

Always glad to help. You provide a great plugin and reporting on the debugging and such I have to do anyway is the least I can do.

EDIT: Just to be clear, commenting out these lines from the role scoper file will fix the issue:

7         if ( function_exists( 'relevanssi_build_index' ) )
8             add_action( 'shutdown', array( 'Relevanssi_Admin_Helper_RS', 'do_reindex' ) );

Looked into the role-scoper code a bit more. Every time an object is saved it looks for the existance of ‘relvanssi_query’, then calls rvi_index() which hooks relevanssi_build_index() to ‘shutdown’ (called every time PHP execution is about to end). That’s roughly the chain of events leading the problem. Looked into the Role Scoper forums and found that this may be related to relevanssi indexing post excerpts and such which then allows for users to see content that Role Scoper would typically not allow since it is held in the index and not filtered through the usual search results. I’m not delving much further into the issue as we really use Role Scoper for editor roles and not public so this does not apply for us. I will continue to keep up with this thread and look forward to a solution that doesn’t require me to remember to comment those lines out every update ??