[mapserver-users] Rendering Hindi Characters

Wed Jan 23 17:05:32 PST 2013

FYI, Mapserver does not do anything for rendering text except:

1. pass the text to fribidi which converts a utf8 sequence of characters 
into a "displayable" sequence of characters.

2. mapserver then passes that to the text rendering function of whatever 
renderer you are using.

Mapserver does not try to understand it, it totally relies on the 
underlying rendering code to do the right thing whatever that is.

Thomas Bonfort might have some more insight into this as he re-engineer 
a lot of the rendering pipeline and it might be helpful to know what 
code is doing the actual text rendering. It used to be freetype and may 
still be.

I'm also not convinced the fribidi is not the right place for this to 
happen regardless as it is responsible for taking a sequence of 
characters and generating an "displayable" sequence that can be rendered 
correctly given the correct font.

-Steve

On 1/23/2013 7:17 PM, Mr. Puneet Kishor wrote:
> Peter,
>
> My guru in all things Hindi fonts tells me that the problem lies at the level where encodings are converted into font renderings. Apparently, र् ट is a perfectly correct alternate rendering of the unicode, just not Correct (with a capital C). I have no idea where this happens with MapServer. Thankfully I've never had to create a map with Hindi labels (I would be curious to learn, perhaps off-list, who wants to see maps with Hindi labels), but I too would like to see this resolved.
>
> Hope someone knowledgable chimes in.
>
>
> On Jan 23, 2013, at 3:58 PM, "Mr. Puneet Kishor" <punk.kish at gmail.com> wrote:
>
>> (adding the list back to cc so brains better than mine may help)
>>
>>
>> On Jan 23, 2013, at 3:47 PM, "Peter Mallen" <peter.mallen at airborne.aero> wrote:
>>
>>> Hi Puneet,
>>>
>>> I have passed the codes you have provided into MapServer  (U+0930 U+094D U+091F ) and it displays the characters separately, not as one character.
>>>
>>> This is how Map Server displays it: र् ट
>>
>>
>>
>> hmmm... well, then this is definitely above my pay grade. Seems like you are doing the right thing, but my guess is MapServer's Unicode implementation is not working properly.
>>
>> I don't see any mention of Unicode on mapserver docs page. The only potentially relevant bit I see is  (http://www.mapserver.org/installation/unix.html#obtaining-the-necessary-software) that libgd should be compiled against the iconv library which provides full support for Unicode. On my MBA I get
>>
>>     $ ~ > gdlib-config --libs
>>     -lXpm -lX11 -ljpeg -lfontconfig -lfreetype -lpng -lz -liconv
>>
>>
>>
>>>
>>> Thank You,
>>>
>>> Peter Mallen
>>> -----Original Message-----
>>> From: Mr. Puneet Kishor [mailto:punk.kish at gmail.com]
>>> Sent: Wednesday, January 23, 2013 3:36 PM
>>> To: Peter Mallen
>>> Subject: Re: [mapserver-users] Rendering Hindi Characters
>>>
>>>
>>> On Jan 23, 2013, at 3:00 PM, "Peter Mallen" <peter.mallen at airborne.aero> wrote:
>>>
>>>> Hi Puneet,
>>>>
>>>> I did try feeding MapServer the correct Unicode characters, and it did fix the characters using what I believe is called the "Reph" symbol (ि) but I could not get the conjunct consonants to display correctly.
>>>
>>>
>>> I have no idea what the above means ;-), but, did you try passing U+0930 U+094D U+091F to MapServer? See http://www.unicode.org/charts/PDF/U0900.pdf
>>>
>>>
>>> U+0930 -> र -> r
>>> U+094D ->  ् -> "halves" the previously typed r 091F -> ट -> thick t
>>> U+sound
>>>
>>>
>>>
>>>>
>>>> Also, I have posted on the fribidi list and I believe it does only help with bidirectional languages, but I could be wrong.
>>>>
>>>> Thank You,
>>>>
>>>> Peter Mallen
>>>>
>>>> -----Original Message-----
>>>> From: mapserver-users-bounces at lists.osgeo.org
>>>> [mailto:mapserver-users-bounces at lists.osgeo.org] On Behalf Of Mr.
>>>> Puneet Kishor
>>>> Sent: Wednesday, January 23, 2013 12:27 PM
>>>> To: mapserver-users at lists.osgeo.org
>>>> Subject: Re: [mapserver-users] Rendering Hindi Characters
>>>>
>>>>
>>>> On Jan 23, 2013, at 12:17 PM, Stephen Woodbridge <woodbri at swoodbridge.com> wrote:
>>>>
>>>>> On 1/23/2013 2:20 PM, Peter Mallen wrote:
>>>>>> Hello Everyone,
>>>>>>
>>>>>> I am having issues rendering certain Hindi characters. I am using
>>>>>> UTF-8 encoding.
>>>>>>
>>>>>> When attempting to display the following Hindi character in
>>>>>> MapServer: र्ट
>>>>>>
>>>>>> It is rendered in MapServer as the following two characters: र् ट
>>>>>>
>>>>>> I believe these characters form what is called a conjunct consonant,
>>>>>> of which there appear to be thousands of and most of them do not
>>>>>> render using MapServer.
>>>>>>
>>>>>> Also, It appears that MapServer is having an issue displaying these
>>>>>> conjunct consonants and wants to display these characters separately
>>>>>> rather than to form them as one combined character.
>>>>>>
>>>>>> Here is a link describing more about the Devanagari alphabet:
>>>>>> http://www.omniglot.com/writing/devanagari.htm
>>>>>>
>>>>>> Can anyone please help with this?
>>>>>>
>>>>>> Thank You,
>>>>>>
>>>>>> Peter Mallen
>>>>>
>>>>> Hi Peter,
>>>>>
>>>>> I can not speak to this specific problem but I can give you and overview. Mapserver uses fribidi library for rending this data and I know from working with Arabic that a lot depends on the font you are using, in additional to the capabilities and version of fribidi library.
>>>>
>>>>
>>>>
>>>> Isn't fribidi specifically for bidirectional languages? Hindi is strictly LTR.
>>>>
>>>>
>>>>>
>>>>> This kind of question is probably best asked on the fribidi list as they will know more about what is supported or not for any given language and they might be able to point you to a better font.
>>>>>
>>>>> http://www.fribidi.org/
>>>>>
>>>>> You should probably make sure you are using a recent version of the library like 0.19.2 or better and be aware the 0.19.5 was released in Dec 2012.
>>>>>
>>>>
>>>>
>>>> Peter, I forget where you and I left our off-list conversation re. the issues you were having with rendering Devnagari. Did you try feeding MapServer the correct Unicode characters?
>>>>
>>>> Hope you figure this out.
>>>>
>>>>
>>>> --
>>>> Puneet Kishor
>>>
>>
>
> _______________________________________________
> mapserver-users mailing list
> mapserver-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapserver-users
>