Correctly Displaying UTF8 letters

[Joshua Haynes]

Hi all -

I'm building a market information system here in Niger using http-requests, php, mysql and some FrontlineSMS keyword love. In two of the natural languages of the system, there are special letters that need to be displayed. [Hausa: Ɓɓ, Ɗɗ, Ƙƙ, Ƴƴ and Zarma:Ɲɲ and Ŋ ŋ] .

I am able to place the html entities of the letters (eg. "Ɲ"="Ɲ") in the db, to recall the letter using my php file and to display the letters correctly, using "html_entity_decode($foobar, ENT_NOQUOTES, 'UTF-8');", in the source code of the php page. However, when the letters get passed back to FrontlineSMS, they are displayed in the ANSI format (Ƴ instead of Ɲ).

Am I missing something somewhere? Has anyone else run into this problem?

Kind regards,

Joshua

[Alex Anderson]

Hi Josh, how are you?

Sorry you're having problems with your Hausa, but thanks for the detailed report. I'm guessing you're running on a Windows machine? I agree this looks like a character encoding problem, but it's worth checking if it's with your database rather than with FrontlineSMS.

I added these special characters to my FrontlineSMS transalation files, and some of them seem to display okay - check out the screenshot.

Hope this helps,

Alex

[Joshua Haynes]

Hi Alex, I'm great; good to hear from you!

Yes, for what it's worth, I'm running on Windows. I've included some screenshots, which I should have done yesterday, to demonstrate what is happening:

1-db: in the 'h' column is the html entity and in the 'z' column is the encoded letter, the charset is utf8_unicode_ci for everything.

2.-php source: is the source code of the response for querying both the h and the z. you can see that the h is correctly displayed while the z is all '?'s. (this is normally how i've been verifying what will be sent to the phone, ie what's in the source code gets passed to the $command_response)

3.-frontlinesms: this shows the response when requesting the 'h' column that is sent out to the phonse. i verified in notepad++ that this is the ANSI coding of the UTF8 that should be displayed.

So if I can get the correct letters all the way to my source code for the message, do you think it's a database encoding issue.

I also attempted sending a message direct from frontline with all of the special characters. The same boxes appears as you showed in your pic (why is that?), but on the phone, all the characters were displayed correctly except for Ɲ ɲ. (This is a Nokia thing, I presume).

Thanks for any additional help!

Joshua