evidence on how *slow* and inefficient qTranslate is … please do something

I’m going to use this thread to post code optimizations that I do to qTranslate for anyone who is interested.

It’s not actually necessary to seek translations for ALL the options in the wp_options table. Most of them are only used internally and do not require translation so the whole process of seeking a translation can be cut down.

Many WP installations have a huge wp_options table due to a large number of plugins — it doesn’t matter if they are disabled or not, their options are still there in the table. Most of them ‘forget’ their options in the table even if you uninstall them so if you had previously tried a large number of plugins in the past and uninstalled most of them, your website can get increasingly slow if qTranslate is enabled due to the code below. My wp_options table has 491 rows!

Commenting out the following block in qtranslate_core.php makes the website a hell of a lot faster for me:

/*
if(!defined('WP_ADMIN')) {
	$alloptions = wp_load_alloptions();
	foreach($alloptions as $option => $value) {
		add_filter('option_'.$option, 'qtrans_useCurrentLanguageIfNotFoundUseDefaultLanguage',0);
	}
}
*/

With this tweak, the page from the initial post loads in under 1.5 seconds now, down from 5.5 seconds!

In case you have options that need to be translated, then you can manually hook the qtranslate filter above only for those options, rather than hooking into all options, e.g. I would replace the above code with

$options = array('options','that','need','translation','here');
foreach ($options as $opt)
    add_filter('option_'.$option, 'qtrans_useCurrentLanguageIfNotFoundUseDefaultLanguage',0);

And you can add that into your functions.php.

You can also make the qtrans_parseURL() function faster, in qtranslate_utils.php. This function is called quite a few times:

function qtrans_parseURL($url) {
    $r  = '!(?:(\w+)://)?(?:(\w+)\:(\w+)@)?([^/:]+)?';
    $r .= '(?:\:(\d*))?([^#?]+)?(?:\?([^#]+))?(?:#(.+$))?!i';

    preg_match ( $r, $url, $out );
    $result = @array(
        "scheme" => $out[1],
        "host" => $out[4].(($out[5]=='')?'':':'.$out[5]),
        "user" => $out[2],
        "pass" => $out[3],
        "path" => $out[6],
        "query" => $out[7],
        "fragment" => $out[8]
        );
    return $result;
}

Even though the above could be sped up by using the S PCRE modifier to analyze the regex for future calls, you can replace the entire function with the much faster:

function qtrans_parseURL($url) {
        $result = parse_url($url) + array(
                'scheme' => '',
                'host' => '',
                'user' => '',
                'pass' => '',
                'path' => '',
                'query' => '',
                'fragment' => ''
        );
        isset($result['port'])
                and $result['host'] .= ':'. $result['port'];
        return $result;
}

note the + array union operator which basically implements default array values since parse_url() populates those array fields only if a match was found in the url.

Looking through the code, I’m cringing at the sight of stuff like this:

elseif(preg_match("#^<!--:-->$#ism", $block, $matches)) {
   $current_language = "";
   continue;
}

The above is a clear example of what I was talking about. Not only $matches is not used anywhere, so preg_match() shouldn’t have been asked to create and populate a new array, and not only are the s, m and i modifiers not needed, but the entire if clause could have been replaced with just:

elseif($block == '<!--:-->'))

There are too many unnecessary calls to preg_match() and preg_replace(), a lot of them shouldn’t even be case insensitive, or multi-line … strpos(), stripos(), str_replace(), str_ireplace() should be used when possible, or plain comparison like above.

Also, there are too many for/foreach loops triggering subsequent large number of calls to preg_match/preg_replace. It’s usually better, if possible, to try and call preg_* beforehand on the big string, rather than split the string and then call preg_* a lot of times.

Hey normadize

I just wanted to stop by and thank you for this thread. I’m having issues with the plugin too: it’s taking up to 60 seconds to load with qTranslate, and about 4 seconds without.

Anyway, I’ve bookmarked this page, and will continue to watch.

Thanks again

db9429

Thank you for your help
I will post some improvements too.
Mainly regarding the way to store the tag translation.
Change 35mo into 48ko…

Hi
I have developed some code to not store the tag translation into the wp_options table option_name qtranslate_term_name field.
It can be improved or stored into the term_meta table, but for my need and with the time I had, I did that.
Basically, it turns 35Mo of serialized array into a 48ko table content.
Here is the related thread https://www.qianqin.de/qtranslate/forum/viewtopic.php?f=3&t=4073
You can remove the unique Key, I did it because I need one language.

CREATE TABLE IF NOT EXISTS wp_qtranslate_terms_lang (
  term_id int(11) NOT NULL,
  term_meta varchar(32) NOT NULL,
  term_value varchar(150) NOT NULL,
  UNIQUE KEY term_id (<code>term_id</code>)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;

Please visit the thread to get the files I updated. Some hardcoding but it can serve as a basis to improve the rest.

Frederic

qtranslate_core.php
– qtrans_saveConfig()
– qtrans_updateTermLibrary()
– qtrans_useTermLib($obj)

qtranslate_wphacks.php
– qtrans_insertTermInput($id,$name,$term,$language)
– qtrans_insertTermInput2($id,$name,$term,$language)

I tried this too https://blog.slo-host.com/2011/10/qtranslate-qtrans_use-recursion-optimization/

https://www.qianqin.de/qtranslate/forum/viewtopic.php?f=3&t=4115&p=12968#p12968

let me know if it works for you

// splits text with language tags into array
function qtrans_split($text, $quicktags = true) {
	global $q_config;

	//init vars
	$split_regex = "#(<!--[^-]*-->|\[:[a-z]{2}\])#ism";
	$current_language = "";
	$result = array();
	foreach($q_config['enabled_languages'] as $language) {
		$result[$language] = "";
	}

	// split text at all xml comments
	$blocks = preg_split($split_regex, $text, -1, PREG_SPLIT_NO_EMPTY|PREG_SPLIT_DELIM_CAPTURE);
	foreach($blocks as $block) {
		# detect language tags
		// Frederic
		// if(preg_match("#^<!--:([a-z]{2})-->$#ism", $block, $matches)) {
		$matches = array();
		foreach($q_config['enabled_languages'] as $language) {
			if($quicktags) { if(strpos($block, "[:".$language."]") !== false) { $matches[1] = $language; } }
			else {
				if(strpos($block, "<!--:".$language."-->") !== false) { $matches[1] = $language; }
			}
		}
		if(isset($matches[1]) && $matches[1] != '') {
			if(qtrans_isEnabled($matches[1])) {
				$current_language = $matches[1];
			} else {
				$current_language = "invalid";
			}
			continue;
		// detect quicktags
		/*} elseif($quicktags && preg_match("#^\[:([a-z]{2})\]$#ism", $block, $matches)) {
			if(qtrans_isEnabled($matches[1])) {
				$current_language = $matches[1];
			} else {
				$current_language = "invalid";
			}
			continue;
		// detect ending tags
		 */
		// Frederic preg_match("#^<!--:-->$#ism", $block, $matches))
		} elseif($block == '<!--:-->') {
			$current_language = "";
			continue;
		// detect defective more tag
		// Frederic
		// } elseif(preg_match("#^<!--more-->$#ism", $block, $matches)) {
		} elseif(strpos($block, "<!--more-->") !== false) {
			foreach($q_config['enabled_languages'] as $language) {
				$result[$language] .= $block;
			}
			continue;
		}
		// correctly categorize text block
		if($current_language == "") {
			// general block, add to all languages
			foreach($q_config['enabled_languages'] as $language) {
				$result[$language] .= $block;
			}
		} elseif($current_language != "invalid") {
			// specific block, only add to active language
			$result[$current_language] .= $block;
		}
	}
	foreach($result as $lang => $lang_content) {
		$result[$lang] = preg_replace("#(<!--more-->|<!--nextpage-->)+$#ism","",$lang_content);
	}
	return $result;
}

I also optimized qtrans_split() and qtrans_use() myself … but the whole thing needs to be rewritten.

The plugin author is not replying to anyone anymore. Maybe it’s time to move on.

I try to see what I can do.

replace in previews post

foreach($q_config['enabled_languages'] as $language) {
			if(strpos($block, "<!--:".$language."-->") !== false) { $matches[1] = $language; }
			if($quicktags && strpos($block, "[:".$language."]") !== false) { $matches[1] = $language; }
		}

A first (not-complete) optimization for qtrans_use() which calls itself unnecessarily many times is as follows. Replace the lines:

global $q_config;
// return full string if language is not enabled
if(!qtrans_isEnabled($lang)) return $text;
if(is_array($text)) {
    // handle arrays recursively
    foreach($text as $key => $t) {
        $text[$key] = qtrans_use($lang,$text[$key],$show_available);
    }
    return $text;
}

if (is_object($text)||@get_class($text) == '__PHP_Incomplete_Class') {
    foreach(get_object_vars($text) as $key => $t) {
        $text->$key = qtrans_use($lang,$text->$key,$show_available);
    }
    return $text;
}

// prevent filtering weird data types and save some resources
if(!is_string($text) || $text == '') {
    return $text;
}

with

if (empty($text) || !is_string($text) || !preg_match($re = '/<!--:[a-z]{2}-->/', $text) || !qtrans_isEnabled($lang))
        return $text;

if (is_array($text) || $text instanceof __PHP_Incomplete_Class) {
    foreach ($text as &$t)
        if ($t && is_string($t) && preg_match($re, $t))
            $t = qtrans_use($lang, $t, $show_available);
    return $text;
}

global $q_config;

Note the & in the foreach loop.

Actually, most of the above is not needed. In fact, the second if (is_array() ... is not even reached if $text is an array or object, which is Ok in most cases, see below. You could just as well remove it altogether.

The qtrans_postsFilter() function is hooked into ‘the_posts’ filter. This function is:

function qtrans_postsFilter($posts) {
	if(is_array($posts)) {
		foreach($posts as $post) {
			$post->post_content = qtrans_useCurrentLanguageIfNotFoundShowAvailable($post->post_content);
			$post = qtrans_useCurrentLanguageIfNotFoundUseDefaultLanguage($post);
		}
	}
	return $posts;
}

Note how it is passing each object $post in a loop to qtrans_useCurrentLanguageIfNotFoundUseDefaultLanguage($post) which in turn calls qtrans_use() on each of the object’s properties. Only the title and content (and maybe guid and pinged) properties require translation (none of the other 20+ do) and for those, qTranslate already hooks into ‘the_content’, ‘the_title’ etc filters. This function can be useful if you call get_posts() in your own code.

You could remove the ‘the_posts’ hook from qtranslate_hooks.php and if you need to call get_posts() in your own code and want the title and content translated, then you can either call qtrans_postsFilter(), which is very inefficient (and you may not want the default behaviour of not showing the default language for the content if the current language is unavailable), of just translate the title and content in a loop — you can have your own function for that.

Thank you normadize.
I`m working currently on it.
Patch, hardcode etc, and still slow.
I continue.
Also qtranslate slug can maybe be improved.
I check qtranslate currently.
Thanks

To keep current functionality but with a faster, less-recursive function, replace in qtrans_use() the following:

global $q_config;
// return full string if language is not enabled
if(!qtrans_isEnabled($lang)) return $text;
if(is_array($text)) {
    // handle arrays recursively
    foreach($text as $key => $t) {
        $text[$key] = qtrans_use($lang,$text[$key],$show_available);
    }
    return $text;
}

if (is_object($text)||@get_class($text) == '__PHP_Incomplete_Class') {
    foreach(get_object_vars($text) as $key => $t) {
        $text->$key = qtrans_use($lang,$text->$key,$show_available);
    }
    return $text;
}

// prevent filtering weird data types and save some resources
if(!is_string($text) || $text == '') {
    return $text;
}

with the following

if (empty($text) || !qtrans_isEnabled($lang))
    return $text;

$re = '/<!--:[a-z]{2}-->/';
if (is_string($text) && !preg_match($re, $text))
    return $text;

if (is_array($text) || is_object($text)) {
    foreach ($text as &$t)
        if ($t && ((is_string($t) && preg_match($re, $t)) || is_array($t) || is_object($t)))
            $t = qtrans_use($lang, $t, $show_available);
    return $text;
}

global $q_config;