[Plugin: Yet Another Related Posts Plugin] cache rules cause thrashing in 3.1.3

YARPP can cause thrashing and high CPU usage under certain conditions. In large WP blogs (thousands of posts) these can cripple the server. YARPP has a cache mode that greatly improves performance, but under these certain conditions, thrashing occurs.

This is YARPP 3.1.3 (and 3.1.4 beta)

Here is the detail:

I have a post, ID 9812. YARPP finds 3 related posts for it.

The info gets cached and you can see that 9812 has three IDs associated with it in the cache:

mysql> select reference_ID,ID from wp_yarpp_related_cache order by reference_ID;
+————–+——+
| reference_ID | ID |
+————–+——+
| 9812 | 0 |
| 9812 | 7761 |
| 9812 | 8032 |
| 9812 | 8049 |

Now I go browsing around as a user (not admin) looking at some other posts and the cached info stays the same…

UNTIL I view some post where YARPP finds that good old 9812 is related to the newly-browsed post.

YARPP deletes the info from the cache table for 9812, and instead we just have an indicator that some posts “point to 9812” but 9812 no longer points to any other posts:

+————–+——+
| reference_ID | ID |
+————–+——+
| 3548 | 9812 |
| 4226 | 9812 |
| 6351 | 9812 |
| 7987 | 9812 |
| 8764 | 9812 |
| 9708 | 9812 |

No author has changed anything in the system in the meanwhile. Nothing has changed except that some user has browsed to a page that YARPP determines is related to 9812. YARPP has deleted the 9812 info.

So 9812 now has to have its YARPP cache rebuilt for the next site visitor. In my case, this involves 2 MySQL processes and about 15 seconds of CPU time because of the size of the blog!

In sites with large numbers of posts (this one has thousands), and where the last few posts might all be somewhat related, YARPP can end up causing a lot of churning because it’s so frequently deleting the posts, even though no posts have actually changed.

Here’s what happens when the visitor browses back to 9812 – showing that the cache entries had to be re-created again.

+————–+——+
| reference_ID | ID |
+————–+——+
| 3548 | 9812 |
| 4226 | 9812 |
| 6351 | 9812 |
| 7987 | 9812 |
| 8764 | 9812 |
| 9708 | 9812 |
| 9812 | 0 |
| 9812 | 7761 |
| 9812 | 8032 |
| 9812 | 8049 |

I wonder if maybe YARPP doesn’t have to delete 9812 at all just because it discovered that some other posts think THEY are related to 9812.