EasyfullsearchWP

Description

EasyfullsearchWP replaces the native WordPress search engine to improve the effectiveness and relevance of all searches.
It lets you manage posts an/or pages, to choose the fields you want to use in the searches (including comments, author, custom fields, categories, taxonomy …).
By giving you predefined values, it helps you to easyly improve your user’s experience.

As the easyfullsearch engine is more powerful than the basic wordpress search mechanism. it allows administrator and developer to use the wide range of features of the easyfullsearch engine.

This version is the free version of EasyfullsearchWP. Presently, easyfullsearchWP is managing main user’s searches coming from the search widget or administation’s pages.

Easyfullsearch has a companion tool, called Efsmyadmin. This tool gives you access to a large amount of functions to manage the index. It is available on the Easyfullsearch’s web site (https://www.easyfullsearch.com).

Key features

• Results are sorted in the order of relevance according to configurable weights.
• A very fast retrieving process.
• You can choose to have a strict matching (all the words) or flexible (at least one of them.) This may be an automatic option.
• The fuzzy logic may be used to match results (never, always or only if the strict match is not fruitful). The fuzzy words are extracted from the content itself.
• The date can be used as complement of the sort algorithm.
• You can decide to use search for posts and/or pages.
• You can choose to search in different fields : title, content, comments, tags, categories and custom fields.
• You give your own weight to each fields
• A full documentation is available directly through the settings menu.
• You can highlight the relevant terms

Advanced features

• A very fast indexing process.
• You can search only in fields which are pertinent for you (for example you can exclude the comments or the title !).
• You can choose to use the Easyfullsearch for the users’ searches or administration’s searches.
• You can choose the types of posts to search (post and/or pages)
• The search on each types can be easyly turned “on” or “off”.
• You can choose the weighting for all types of fields. You don’t need to rebuild the index after changing them.
• You can optimize the index for the most used words.
• You can log the users’ requests, and see what exactly happened in your site.
• Custom post types and custom taxonomies can be used and weigthed as the others fields.
• EaysfullsearchWP supports the polylang plugin
• Easyfullsearch supports site with huge base and hundreds of thousands of posts.
• EaysfullsearchWP lets you easyly manage the stop words (create, suppress, add a language …)
• The plugin automaticaly limits the disk space used for logs
• EasyfullsearchWP uses queued inserts to protect the index against inconsistency.

Coming

• Enhance statistics
• Manage automatic suggestion. The autocomplete is already part of the engine.

Translations

• English – default, always included
• French – toujours disponible

Settings

The plugin’s settings are managed through an unique administration page. This page is part of the “settings” menu, in administration interface of the site. The administration page is nammed “Easy Full Search”.

This page is composed of different sections (all these points are discussed in later sections :

• “Main actions” : Contains the basic actions to set up the plugin
• “Index values” : Gives informations about state and values of the index
• “Search : post types” : Allows to choose which types of elements are to be included in the search results
• “Search : fields and sort” : Lets choose weight and sort’s order of each elements in the search result
• “Search : sort with date“: Lets choose the way that the dates are used as criteria in the results
• “Search : operators” : Lets you decide the operators used in the regular search
• “Stop words” : Manages the list of stop words according to the language
• “Change current user” : Lets you change the current user and its password.
• “Miscellaneous” : Others actions

This part describes in detail all the setting sections.

Main concepts

In order to understand options and actions available in the setting page, some concepts of Easyfullsearch should be explained .

Easyfullsearch (also called EFS) is a generic search engine for full text and faceting searches. It is written in php language. Its goal is to have a modern and powerful search engine, fully integrated in the php environment. Easyfullsearch is self sufficient. It doesn’t use any external database.

Easyfullsearch uses some basic concepts :

• “index” : the place where Easyfullsearch stores all the data used to deliver the service
  Easyfullsearch manages its own index (a collection of files). It doesn’t use generic databases (such as mysql). An index contains all files needed to manage queries.
  The index is a subdirectory of the “efs_data” directory (which is directly under the easyfullsearchWp directory). Easyfullsearch could manage multiple index. in the case of wordpress, only one index is created. The name of the index (the directory itself) is based on the name of the site.

  It means that if you remove the plugin, the index will be also remove. It also means that you can save all the index by saving its directory. You also could move the index from one server to another by simply moving this directory. In this case you must also be sure that wordpress’ references (posts’ ids ) are unchanged. The index uses the wordpress’ ids as item references.

• “item” : All entities stored in an index are called item. An item is a collection of fields. This generic term is use to show that an item could be a set of words, posts, pages, classified … and can be more complex than a simple document. In the case of wordpress, an item is a post or a page. This means results may contain post or page references.

• “Item id” : A unique id to identify an item. In the case of wordpress the item id is directly the wordress id associated to the post or the page.

• “Field” : The fileds are the different parts of an item (such as title, content, date …). Fields are containing terms. In the case of wordpress mains fields associated to an item are : title, content, comments, author, attached files, dates …

• “Term” : The basic element in a field (in general a word, a date …)

Basically, in the wordpress usage, the “items” are posts or pages, the “fields” are the different parts of the item (such as post_title, post_content …), and the “terms” are words extracted from the fields.

That’s enough for concepts. More explanation can be found in the easyfullsearch website (https://www.easyfullsearch.com)

Section : “Main actions”

This section contains the most useful actions to manage the plugin. In most of cases, you only need these.

Button : “Update the index”

• This button is useful if your web site contains items befoe you install the easyfullsearchWP plugin. As the index is empty at the activation, you have to index all already existing items (pots, pages). It’s also the case if you remove all items from the index (button “Empty the index” ).

  This action will search in the wordpress base all the items (posts, pages …) to index. You can give a large amount of items to index in one pass. A default value is generated according to the capability of your server to process the operation. The suggested value is “safe”. Easyfullsearch has a great capability of fast indexing.

  easyfullsearchWP memorise the last id indexed (see informations section). If all item are not indexed, the process continue from the last indexed item. If all items can’t be indexed in one pass, you need to repeat the operation. The value suggested will be zero if Easyfullsearch finds that all items are indexed. in general site with less than 20 000 ietms will take few minutes to index all the base. If you have very large bases and need some help, let me know.

  All posts and pages created or modified after the activation of the plugin will be automatically indexed.

  If your wordpress site have no page and no post awhen you activate the plugin, you have nothing to do. All pages or posts will be automatically added to the index when they will be published.

Option : “Use easyfullsearch for users’s searches”

• When you activate the plugin, easyfullsearch will not automaticaly replace the standard wordperss search mechanism. As discussed before, some posts or pages may be not indexed at that time. To use actually the plugin in user’s searches you have to activate it by checking the check box of this option.

  It allows you to temporary use the standard wordpress search. You don’t have to desactivate the plugin to do that. Even if this option is not checked, Easyfullsearch continues to index all the created or modified items.

Option : “Use search for admin”

• When you activate the plugin, easyfullsearch will not automaticaly replace the standard wordperss search mechanism in the administration menu. As for users’ searches, some posts or pages may be not indexed at that time. To use actually the plugin in the administration menu, you have to activate it by checking the check box of this option.

  It allows you to temporary use the standard wordpress search. You don’t have to desactivate the plugin to do that. Even if this option is not checked, Easyfullsearch continues to index all the created or modified items.

Button : “save options”

• This button will save main options. (see above).

Section : “Index values”

This section gives basic useful information about the index. All information is given to help or satisfy curiosity. In general you doesn’t really need them.

Information : “Index name”

• The name of the index. It have been generated during the installation. It is composed with the prefix WP and the name of your website. The result is something like “WPmysite”.

Information : “Index Dir”

• The path of the directory containing the index. All files of the index are stored in this directory (with subdirectories). The regular name is : mysite\wp-content\plugins\easyfullsearchWP/efs_data/index_WPmysite.

  As you see easyfullsearchWP created a subdirectory called efs_data. The actual directory of the index is “index_WPmysite”.

Information : “Created date”

• The date of creation of the index. Actually it is the first activation of the plugin.

Information : “Last update”

• The last time the index have been updated (means an item has been created, updated or deleted).

Information : “Estimated indexed items (posts,pages)”

• This is the estimated number of items in the index. In some cases this can be different from the reality of a few units)

Section : “Search : post type”

This section lets you manage the type of elements to include in the search.

The list of indexed elements, associated to checkbox, allows you to manage each of them. The fact to check or not a type doesn’t means you need to rebuild the index.
You can manage the fields used in the search itself in the next part of the options.

Option : “Post”

• If checked, queries will search in posts’ content.

Option : “Page”

• If checked, queries will search in pages’s content

Section : “Search : fields and sort”

This section is important. Default values have been defined to insure the maximum quality of results. A more precise tuning may be usefull depending the content itself. To correctly manage this part, you need to understand how results are sorted.

When you change the weight of fields, the next searches will be execute according to these values. It’s not necessary to rebuild the index. So you can try different options to find the best values.
To present how easyfullsearch sorts the results, you have to consider two axes. The first is the weight of each field in the sort. The second is the fact to separate different fields.

In general search engines combine both by only using a weight attached to each field, the common manner is to give a higher weight to the most important field. But this way introduced a few unexpected sorting results or required to have a smaller range of values for less important fields, giving sometimes confusing results.

The approach used by easyfullsearch is more powerful, even if we only use a part of it in this plugin. Basically, we have two “sorting methods”.

The first is an associated calculation of occurrence and fields. For each selected field, the number of occurrence is multiplied by the weight associated to the field. This mechanism lets you decide the combination of weight. If you want to favour one field, you just have to give it a large weight. If you want to have more mixed results, you can give a narrower range of weight. The default values are set up to give priority to the title (weight of 10000) but if another item contains a lot of occurrences in the content field (weight of 1000, it can have the same final weight.

The second one is the date (day/month or year) of the last updated of the post or the page. The value of the date is managed by wordpress. The choose of “day” as smaller value have been done to limit the effect of date in the result. The use of date may be more time consuming if you have a large amount of results. You must consider the fact that this option in unnecessary in general.

This section contains the list of fields which could be used or not, and the weight associated to each of them. Let consider the first column

The first column lets you choose which fields will be used in the search proccess. You could change it as you like without re-indexing. This will be directly use in the next search (user or admin).

The fields are applicable as well to the posts or pages

• title : search, or not, in the title of item
• content : search, or not, in the content of item
• comments : search, or not, in the comments associated to an item. The EasyfullsearchWp plugin aggregates all comments text as only one. This new created field is part of the element
• author : search, or not, in the author name associated to the item
• tags : search, or not, in the tags associated to the item
• taxonomy : search, or not, in the taxonomy values associated to the item
• custom_content : If custom fields are are defined, search, or not, in their content. The EasyfullsearchWp plugin aggregates all these contents as only one. This new created field is part of the element.
• attachment_text : search, or not, in the text associated to item’s attachment (such as the name of a file). Presently, this doesn’t allow to search in the attachment content itself (may be next …).

The second column gives the weight associated to each field. If you want to change default values.You need to understand how the results are sorted to well manage this part. See explanations above.

If you give a value of 0 to a field, it just give it a null weight. This doesn’t mean that the field is not used in the search. It means that this field is not important in the sorting process. So, a post could be part of the result even if one occurence have been found only in this field. It will simply placed at the end of results. To exclude a field, use the previous options in the first column.

You could change the weights as you like. The next searches will be executed according to these values ( without rebuilding the index). In general, you have to try different options. As next searches will use the new values, you can directly test them. Easyfullsearch offers you a large liberty to establish your own balance.

Section : “Search : sort with date”

After establishing the use and the weight of each field, you could choose to also considering the date in the sort process. In this case you can choose to give priority to the date compared to the fields. the date could be used before (means the newer item will be classified first) or after (means that if two items have the same weight, the newer one will be classified first).

Option : “Use the date as a sorting criterion”

• If checked the date is part of the sorting process. The use of date may be more time consuming if you have a large amount of results

Option : “Use the date criterion before the weights”

• As described above, if checked, results will be sorted using date criterion before those of weights associated to the fields.

Option : “Choose the period of the date for sorting”

• This option lets you manage the date period used as criterion. The avalaible periods are day, month and year.

Section : “Search : operators”

Option : “Use AND as default operator”

• This options lets you to choose the default operator if the search string contains more than one term (“AND” or “OR”). The alternative is to search all of the terms (the AND operator, the default value), or at least one of them (the OR operator). Lets consider the “Vinci code film” search.

  If you check the AND option , the query will be interpreted as “Vinci AND code AND film”. This way may be more natural, but it is also more restrictive. You could be desapointed not seeing some results.

  If you let the option unchecked,the “OR” operator will be used. In this case the query will be interpreted as “Vinci OR code OR film”. On one hand, you will have more results, on the other hand the results may contain posts or pages less relevant.

Option : “Use OR operator if no results”

• If you choose the ‘AND’ operator as default, you can check this option to indicate to EasyfullsearchWP to try the ‘OR’ operator if no results are returned after the search. In fact if no posts or pages contain “Vinci AND code AND film”, Easyfullsearch will try to find some results for “Vinci OR code OR film”.

Option : “Use fuzzy query “

• Easyfullsearch engine is able to generate “fuzzy” searches. The terms are taken from the dictionnary of the actual content of the index (not from the requests). According to the last generation of dictionnaries, easyfullesearch try to find approching terms and to include them in the search. This option is powerful but it could be time consuming. if many “fuzzy” terms are used. This could slow your site.
  it needs that the the dictionnary have been generated. The more recent it is, the more relevant may be the fuzzy logic.

Option : “Default language”

• The default language is used by Easyfullsearch when a new item is indexed. During the terms extraction, Easyfullsearch will use a stop words list to exclude some terms (words) from the index. These terms will not be found in searches later. The index will be less space consuming, faster during the inserts and the searches. You can manage different lists of stop words according to the language. This option gives you the way to choose the language used.

  Some stop words list are embedded within the plugin (typically French and English). If you need to add another language you have first to set up the new language (the next option field named “New language”. You must save options before changing the new list of stop words.

  If a language is associated to the item to index (such as a post), it will be used as language for the indexing operation. The default language is only used if no language is associated to an item. The language is supposed to be associated with the taxonomy field.

  As language is used in the index operation to take the correct list of stop words, it is not necessary to distinguish different use of the same language, such as French in Belgium, Canada or France, or English in UK or USA. This is useful for translation, but not really for stop words.

Entry : “New language”

• If you need to add a new language to support a new list of stop words, you can define it here. It must be saved before updating the list. In fact the value can be any (no syntax associated to the language). It means you also could define your own space of words to manage different behaviours.

Button : “Save search options”

• The button Save search options saves all options related to the search, from “post types” to “search operators”. Some values are updated in the wordpress context only (like the posts type). Some others values are directly update in the configuration of the Easyfullsearch engine (such as logging users’s requests).

Section : “Stop words”

In every language, some words are useless for the search relevance. The list of stop words allows you to exclude them. It makes the insert/update operations faster as well as the searches itself.

• This section allows you to manage the lists of stop words. The default language associated to the current list ofstop words is indicated just after the section’s title. For example Stop words (en) means the current list is related to the English language.

List of stop words

• All stop words are listed in the text area. They are sorted by alphabetic order. Stop words are separated by the character “-“. Spaces are just added to facilitate the read. If you want to suppress a word from the list of stop words, remove it directly. If you want to add one or more words, insert them in the list separated by the “-” character. No space is necessary. You can insert them anywhere (no sort needed).

Button : “List of stop words”

• This button will updates the list of stop words inside the Easyfullsearch engin. The page will be updated and the list will take account your changes (words are sorted again). All new publishing of posts or pages will use this new list of stop words, but all previously inserted items are not updated.

  All futur search are using the new list. It means that words in the list of stop words will be excluded from the search. In the other hand, it means that all previoulsy inserted items can’t be retrieve if they contain a word newly excluded from the list. In this case you may have to empty the index and rebuild it.

  Section : “Change current user”

Entry : “Current user”

• The current login/password is mandatory to make any change. By default, the current user is “Admin”, completed by a six digits random number. The default password is “efswp”. As this user/password are easy to find inside the plugin, it may be a good thing to change it. On the other hand, this username has a random part and it is used to access to the index which contains only informations also published.

Entry : “Current password”

• The password associated to the current user.

Entry : “New user”

• If you want to change the current user, you can define a new one by giving its name/login here. If this entry is empty, you can change the current password with the next entries. If you want to have more than one user to access to the index, you must use the efsmyadmin tool.

Entry : “New password”

• The value of this entry will be the new password of the current user. If no value is given in the “new user” entry, this will chnage the password of the existing current user.

Entry : “Confirm new password”

• The entry to confirm the value of the new password.

Section : “Others actions”

This section lets you act in the Easyfullsearch index and get some additional infomations

Button : “Optimize the index”

• If your site is often updated, the index can be optimized to ensure a faster answer. In this case, the most often requested terms will be optimized. In most of case you will not have to do that. During this action, publishing may be slowed.

  That’s why the use of this button is protected by the entry field. It demands you to enter a random generated code.

Button “Empty the index”

• This button lets you remove all items from the index. In general you have not to do that. But you have to do if you want to rebuild all the index. This can also be useful if your items are very frequently modified and you want to shrunk the index.

  The use of this button is protected by the entry field after. It demands you to enter a random generated code.

Button “Generate the dictionnary”

• If you want to use the fuzzy logic in the search (see section “Use fuzzy query”), you must generate the dictionnary. The dictionnary is generated from the content of the index. Easyfullsearch will generate “fuzzy” terms from a requests according the nearer terms present in the dictionnary.

  All the whole content of the index is not analyse. Easyfullsearcjh optimizes this action. it will be done in an incremental process. EasyfullsearchWp give you the information how much content is waiting to be to be includes in the dictionnary.

Button “Log / Stop logging requests”

• Easyfullsearch allows you to log the user’s requests. It can be temporarily give you interesting informations. The logs will be available by using the next button “Show log of requests”. This option doesn’t really slow requests. Easyfullsearch has an internal mechanism to limit the amount of data in the log. Older values may be lost.

Button “Show logs of requests”

• If you have logged the users’ requests, you can see them using this button. The next page will show you the list of requests, with :
  • The date of the request
  • The duration of the requests (only for the Easyfullsearch part)
  • The actual search query (may be reused in Efsmyadmin)
  • The number of results returned
  • If any, the part of the query with no results
  • The tool used (the wordpress plugin is nammed : EasyfullsearchWP), the login The tool used (the wordpress plugin is nammed : EasyfullsearchWP), the login of the user (the wordpress username first, then the easyfullsearch username ) and its IP address.of the user and its IP address.

Button “See the documentation”

• This button gives you a direct acceess to the plugin’s documentation.

Plugin’s hooks

The easyfullsearch plugin has several hooks letting developers add functionalities. if you need more explanation on their actual use, please let me know.

The list of the hooks

• easyfullsearchWP_init : Initialize the variables of the plugin.
• easyfullsearchWP_post_save : Insert/modify a post or page in the index.
• easyfullsearchWP_search_istosearch : Determine if the search is to be processed, or not, by the Easyfullsearch engine.
• easyfullsearchWP_search_fieldslist : Change the list of the fields used in the search.
• easyfullsearchWP_search_weightlist : Change the weight associated to each field in order to sort the results.
• easyfullsearchWP_search_posttypes : Change the types of element to search for (post and page)
• easyfullsearchWP_search_optfuzzy : Use, or not, the fuzzy logic in the search process.
• easyfullsearchWP_search_loguserdata : Add some user’s data in the log of the requests.
• easyfullsearchWP_search_results : Modify the results of the search.
• easyfullsearchWP_index_update : Introduce new items in the index.
• easyfullsearchWP_index_add : Create Actually the index.
• easyfullsearchWP_index_model : Set the Easyfullsearch model to use within the plugin.

Documentation of EasyfullsearchWP’s plugin – Version 1.0 – October 2016 – Author M Jarleton – All rights reserved – Copyright Easyfullsearch 2016

Screenshots

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

Installation

Instructions

EasyfullsearchWP is available on the standard wordpress plugins directory.

1. Install the plugin through the WordPress “Plugins” menu directly.
2. Activate the plugin through the ‘Plugins’ menu in WordPress.
3. Go to the ‘Easy Full Search’ settings page and configure it.

You can also download the plugin in zip format and extract files to the “/wp-content/plugins/easyfullsearchWP” directory. Then follow the step 2. (not recommanded)

To update your installation, simply install the new version. You don’t need to rebuild the index.

Activation

In the wordpress plugins page the user must activate the easyfullsearchWP plugin. When activated, the easyfullsearchWP plugin manages several actions :

At this moment, it will not replace the wordpress search, because the index is empty and your users will have no answers to their requests. Before using Easyfullsearch to actually manage the users’s requests, you must fill the index. EasyfullsearchWP lets you decide when you begin to use it as search engine.

After activating, all new posts or pages, as all updates, will be automaticaly indexed by easyfullsearch. All data created before the activation have to be indexed if you want that your users could access them.

So, in order to have the search fully operating, you have to fill the index (see the “Main actions” section of this guide).

Requirements

• WordPress 3.1
• PHP 5.4. Go faster ! : EasyfullsearchWP is php7 compliant.

Deactivation

If you deactivate easyfullsearchWP, it will no more replace the standard search of wordpress. The index itself is not delete and all the data are still present, but any additional updates or creations will not be indexed. So, if you want to activate it again, after creation or update in your posts or pages, the best way is to empty the index and rebuild it again.

FAQ

I have no results in the search.

• If you just have activate the plugin, easyfullsearcWP has just created an empty index. So, it cannot find any results for your queries. The good news is that if you have no results, it means the plugin is correctly activated.

  • To fill up the index go to the Settings (Easy Full Search page in administration) and use the button “Update the index” to fill up the index. While the index is not totaly filled up; you can unchecked the option “Use easyfullsearch for users’s searches”.
  • Check the fields included in the search (“Search : fields and sort” section). May be have you excluded some important fields such as content or title.
  • Check if post type is correctly selected in the section “Search : post types” . You also could try to manually index one post, containing a specific pattern, in order to test the indexing and the search process.
  • You can also try to empty the index and rebuild its.
  • If the problem is still there, after trying these actions, let me know.

Which posts or pages are stored in the index ?

• All published posts and pages are stored in the index, even if you don’t use them in the search. By this way, if you want to use them later, no rebuild action is needed.

How many posts could the easyfullsearch support ?

• The Easyfullsearch engine can manage millions of items. The default index of the plugin does support more than 50 000 different terms. this is more than thousands of posts or pages. It will be automatically extended if necessary, but the search may be less efficient. There is no problem to have a larger index. In this case, let me know.

Is easyfullsearchWP supporting multisites wordpress ?

• Presently the plugin is not designed to support the multisite wordpress. This could be done later.

Where is stored the index ?

• The index is stored in a subdirectory (efs_data) of the easyfullsearchWP directory ( Ex. mysite/wp-content/plugins/easyfullsearchWP/efs_data)

Is easyfullsearchWp php7 compliant ?

• Yes – The code of Easyfullsearch is designed to be php 7 compliant. Many tests have been done under php7. The present developments are all done with a php7 version.

if I change the fields that I want to include in the search’s result, do I have to re-index all items ?

• No – all items are stored in the index. You have nothing to do if you choose to use them or not as fields for the searches. The Easyfullsearch index is able manage large index.

Can I put the index on an external server ?

• Easyfullsearch engine is not designed like this in the use of this plugin. We consider than having the index in the same server is more efficient (no network latence, more consistency ..). This is one of the added value of the Easyfullsearch engine. And you have no charges to add for an index server !

Is EasyfullsearchWP managing users’s rights ?

• Easyfullsearch engine includes a users’s rights management. The easyfullsearchWP plugin create its own user to access the index. There is no need to have a link between the wordpress’s users and the easyfullsearchWP one. You can manage this user through the plugin settings.

Is the plugin secured ?

• The plugin respects some good practices for wordpress. Administration page is protected, and rights are checked. Security is an important matter. easyfullsearchWP plugin is delivered “as is”. If you have any suggestion, please let me known.

Is the index crypted ?

• No – Easyfullsearch doesn’t protect specifically the content of the index. The wordpress’s administor have to take security actions to protect it (secured access to the files, for example). Even if the content of the index is not crypted, informations in it are not useful.

Reviews

Contributors & Developers

Changelog

1.0

• The first version of the plugin