Special Characters, not working

Hi arjun212,

Thank you very much for letting me know. I will look into the cause and let you know once I have a solution.

Hi arjun212,

I have found the cause and fixed it. As you mentioned ‘htmlspecialchars_decode()’ was not handling this situation, but by passing ‘ENT_QUOTES’ it correctly handles the single quotes.

I need to make some other adjustments before I push this live, but I hope to update it sometime tonight or early tomorrow at the latest.

Thank you very much for helping us to improve the plugin.

I tried changing the ENT_NOQUOTES to ENT_QUOTES in the get_tweet.php file, to no avail.

The problem i think is that the apostrophe code is &#039, however it is being read as &#39, so it is being skipped by the htmlspecialcharacters_decode().

You can see an example of this on my website : https://www.union.ic.ac.uk/osc/india/

The twitter widget is placed in the footer, and the 3rd, and 4rth tweets contain #&39;.

Thanks

Hi arjun212,

Which part did you change? Could you try using this code:

// Encode Special Chars
$tweet->text = htmlentities($tweet->text, ENT_QUOTES, 'UTF-8');
// Decode HTML Chars like & to &
$tweet->text = htmlspecialchars_decode($tweet->text, ENT_QUOTES);

It appears that the regex to identify hashtags is also catching the code and turning it into a link, which makes it worse. I think the above change *should* turn the code back into a single quote, so it is displayed correctly and not picked up as a hashtag. If not let me know and ill continue working on a fix.

The new versions I made live today contained the above fix, so hopefully updating will resolve the problem. If not let me know and I will continue working on it.

Updated to version 0.6.6, but the single quotes are unfortunately still not being converted.

Can maybe some form of simple preg_replace() be used for this special case. I tried this method, but turns out I do not know regex well enough to attempt it.
I came up with :

//Encode Special Case ' to '
$tweet->text = preg_replace('/[&]\#39;/', '\'', $tweet->text);

Hi arjun212,

You are right it is still bugged. I think it is being re-encoded when stored automatically. So I either need to prevent that or decode when printing to the page (which means I need to avoid the hashtag regex from turning them into a link).

I will work on getting it fixed tomorrow. Sorry this is taking so long and thank you very much for your patience and help troubleshooting the problem. It is really appreciated.

Hi arjun212,

I have found and successfully tested a solution to this. I will be pushing the update out shortly.

However, others still seem to be suffering from character encoding related issues. In light of this I am currently looking into a complete re-think on how I deal with it.

Thank you for your patience.

Scratch that, it still doesn’t work on all sites. Seems like there is some aspect of this dependant on the server PHP version.

I am going to look into dealing with character encoding in a very different way, as currently I think I am making it harder work than needed.

I will be working on this until it is fixed, sorry for this causing so many problems.

Thats fine. Thank you very muh for your support.

Could there be a problem with the twitter api you are using to get the tweets from the server? Would it be possible to change the api used, so it doesnt encode ascii characters.

The only other solution i can think of is possibly creating a php function to look for ascii representations at the display, and convert it before being displayed.

Hi arjun212,

I have just released an update (0.7.2) which I hope addresses this issue. I would be very grateful if you could give it a try and let me know how it goes.

If that doesn’t work I have setup a debugging version which you can get here:

Kebo Twitter Feed – Debug Version of 0.7.2

This will email me the data which is not in the expected format and allow me to resolve the issue quickly.

Sorry for the hassle this issue has caused.

Sorry for the late reply.

But that seems to ahve fixed the problem.

Thanks alot for your help

Hi arjun212,

Don’t be sorry. That you very much for letting me know, I am really pleased that it is working for you now!