Hi @rfischmann
Actually, WP and plugins store almost everything in the wp_posts table. Not only pages and posts but a lot of different records too. For example menu items, service information, custom post types etc etc
WPFTS tries to index all that info, fortunately most of those additional records are short and does not give any sensible index size increment.
Okay
Well, could we try to make a fast check of your database? I have constructed the MySQL query for this, and would ask you to run it in your phpMyAdmin to detect those posts that were indexed wrongly or not indexed.
select * from
(select
p.ID post_id,
wi.id windex_id,
p.post_title,
p.post_type,
p.post_status,
wlog.status,
wd_t.n wdt_n,
dd_t.n_total wdt_total,
dd_t.n_nw wdt_nw,
length(p.post_title) title_len,
(length(p.post_title) / wd_t.n) av_t,
wd_c.n wdc_n,
dd_c.n_total wdc_total,
dd_c.n_nw wdc_nw,
length(p.post_content) content_len,
(length(p.post_content) / wd_c.n) av_c,
(if(isnull(wi.id),128,0) +
if(wlog.status != 3,64,0) +
if((dd_t.n_nw != dd_t.n_total) or (dd_t.n_total != wd_t.n) or (dd_c.n_nw != dd_c.n_total) or (dd_c.n_total != wd_c.n), 32, 0) +
if(((length(p.post_title) > 0) and (wd_t.n = 0)) or ((length(p.post_content) > 0) and (wd_c.n = 0)), 16, 0) +
if(((isnull(wd_t.n) or (wd_t.n < 1)) and (isnull(wd_c.n) or (wd_c.n < 1)) and (p.post_type in ("post", "page"))), 8, 0)) warn_level
from wp_posts p
left join wpftsi_index wi
on p.ID = wi.tid and wi.tsrc = "wp_posts"
left join wpftsi_docs wd_t
on wd_t.token = "post_title" and wd_t.index_id = wi.id
left join wpftsi_docs wd_c
on wd_c.token = "post_content" and wd_c.index_id = wi.id
left join wpftsi_ilog wlog
on wlog.index_id = wi.id
left join (
select
v.did,
sum(if(isnull(w.word),0,1)) n_nw,
count(*) n_total
from wpftsi_vectors v
left join wpftsi_words w
on w.id = v.wid
group by v.did
) dd_t
on dd_t.did = wd_t.id
left join (
select
v.did,
sum(if(isnull(w.word),0,1)) n_nw,
count(*) n_total
from wpftsi_vectors v
left join wpftsi_words w
on w.id = v.wid
group by v.did
) dd_c
on dd_c.did = wd_c.id
) tt
where
tt.warn_level > 0
order by tt.warn_level desc
limit 1000
This MySQL looks scary, but it does simple things: it finds some key data values in WPFTS index and in case something suspicious, it will show the post record in the result with extended data.
The main column to check is “warn_level”. The value there means how worse the data we found.
128 means “post does not have any index record at all”
64 means “indexing was broken because of internal error, status not equal to 3”
32 means “calculated number of words in the document does not equal to actually stored number in index”
16 means “post title or post content has some text data, but number of indexed words is 0”
We mainly interested for records with warn_level greater than 16. Sometimes (often) wp_posts records may have warn_level = 16 because title has only special characters (those are not indexed normally) or content have only images (those not indexed too).
If you found some posts with warn_level = 16, you can check them by post_id in WP Admin to see whether they have indexable text info or not.
Ideally if you can send me CSV or screenshot of this MySQL query result.
P.S. You may need to change “wp_posts” value in the query to actual name of your wp_posts DB table (sometimes wp_ prefix may be different).
Thanks.
