[mapserver-dev] [mapserver-users] Rendering Hindi Characters
Stephen Woodbridge
woodbri at swoodbridge.com
Fri Apr 5 17:27:03 PDT 2013
Thomas, Ian,
Thanks for that, it fills in a lot of background. It also indicates that
we probably need work done to support Hindi that I was not aware of. So
opening a ticket to track this in mapserver is appropriate.
That article also implies that HarfBuzz is still not complete so we
might be trying to deal with a moving target, but at 90% complete it
might be better than what we have now if it is reasonable stable.
-Steve
On 4/5/2013 9:50 AM, thomas bonfort wrote:
> http://behdad.org/text/ is a good read if you haven't already stumbled
> upon it.
>
> --
> thomas
>
>
> On 5 April 2013 15:45, Ian Walberg <ian.walberg at airborne.aero
> <mailto:ian.walberg at airborne.aero>> wrote:
>
> Steve,
>
> I will raise an issue to track this and we will be looking into it
> in more detail.
>
> From the research we have done so far I am not sure that it is an
> issue with fribidi as my understanding that this just addresses
> the l->r and r->l ordering issues.
>
> What we believe is that the mapserver text rendering pipeline
> cannot support the more complex character layouts required in some
> cases for Hinidi and other languages.
>
> Regards
>
> Ian
>
> -----Original Message-----
> From: mapserver-dev-bounces at lists.osgeo.org
> <mailto:mapserver-dev-bounces at lists.osgeo.org>
> [mailto:mapserver-dev-bounces at lists.osgeo.org
> <mailto:mapserver-dev-bounces at lists.osgeo.org>] On Behalf Of
> Stephen Woodbridge
> Sent: Friday, April 05, 2013 5:56 AM
> To: mapserver-dev
> Cc: mapserver-users at lists.osgeo.org
> <mailto:mapserver-users at lists.osgeo.org>
> Subject: Re: [mapserver-dev] [mapserver-users] Rendering Hindi
> Characters
>
> Ian,
>
> As I mentioned before, I think this is an issue with fribidi or
> the font that you are using. I have seen nothing that indicates
> that this is a mapserver problem other than the implied suggestion
> that maybe mapserver should find a different library to solve this
> issue.
>
> Did you post this issue on the fribidi list?
> Did you post the response back here?
>
> I saw that you tried the text in other windows products and it
> works. I can appreciate your frustration with this fact, but this
> does not in itself mean the mapserver is broken. We rely on
> fribidi and the selected font to generate the correct glyphs for
> rendering. If the gylphs are not rendered correctly there are a
> few possibilities:
>
> 1. a bug in mapserver - not likely as we have plenty of working
> examples using multiple other languages 2. a bug/limitation in
> fribidi 3. a font file with missing or broken glyphs.
>
> In most cases of rendering problems that we have seen to date have
> been problems with fonts. The fribidi list has been helpful
> identifying good fonts for specific languages.
>
> If you want to file a bug on the mapserver site, goto
> http://mapserver.org/ and follow the link on the top right of the
> page to the issue tracker. You will need to supply a test case
> that demonstrates the problem and some graphics so non hindi
> readers can see what is bad and what is expected.
>
> Without some more clear evidence that this is a mapserver problem,
> I'm not sure this will get much action.
>
> I am sympathetic, as I ran into a similar issue with Arabic a few
> years ago and had to do this whole dance with fribidi to
> understand the issues and identify a font that resolved the issues.
>
> -Steve W
>
> On 4/4/2013 11:12 AM, Ian Walberg wrote:
> > Folks,
> >
> > Added the dev list, how do I raise a ticket on this?
> >
> > Thanks
> >
> > Ian
> >
> > -----Original Message-----
> > From: mapserver-users-bounces at lists.osgeo.org
> <mailto:mapserver-users-bounces at lists.osgeo.org>
> > [mailto:mapserver-users-bounces at lists.osgeo.org
> <mailto:mapserver-users-bounces at lists.osgeo.org>] On Behalf Of Ian
> > Walberg
> > Sent: Tuesday, April 02, 2013 2:34 PM
> > To: Stephen Woodbridge; mapserver-users at lists.osgeo.org
> <mailto:mapserver-users at lists.osgeo.org>
> > Subject: Re: [mapserver-users] Rendering Hindi Characters
> >
> > Hello again list,
> >
> > The Hindi rendering problem has come to the top of our
> priorities again and we have been investigating the cause of the
> issue again.
> >
> > To recap the problem we are seeing can be reproduced easily,
> paste Mexico City into google translate and then use the Hindi
> characters with any Unicode font.
> >
> > The top (incorrect) line of text is from mapserver and lower
> > (correct) line is from Excel in this image
> http://tinyurl.com/cs7pax5
> >
> > From the research we have done freetype does not know how to
> render all of these characters together correctly. It does appear
> that pango can be configured to render them correctly.
> >
> > I have not yet dived into the details of the rendering pipeline
> and I hope that someone can jump in and confirm we have a real
> issue and there is not something stupid we have done.
> >
> > Many thanks
> >
> > Ian
> >
> > -----Original Message-----
> > From: mapserver-users-bounces at lists.osgeo.org
> <mailto:mapserver-users-bounces at lists.osgeo.org>
> > [mailto:mapserver-users-bounces at lists.osgeo.org
> <mailto:mapserver-users-bounces at lists.osgeo.org>] On Behalf Of Stephen
> > Woodbridge
> > Sent: Wednesday, January 23, 2013 5:06 PM
> > To: mapserver-users at lists.osgeo.org
> <mailto:mapserver-users at lists.osgeo.org>
> > Subject: Re: [mapserver-users] Rendering Hindi Characters
> >
> > FYI, Mapserver does not do anything for rendering text except:
> >
> > 1. pass the text to fribidi which converts a utf8 sequence of
> characters into a "displayable" sequence of characters.
> >
> > 2. mapserver then passes that to the text rendering function of
> whatever renderer you are using.
> >
> > Mapserver does not try to understand it, it totally relies on
> the underlying rendering code to do the right thing whatever that is.
> >
> > Thomas Bonfort might have some more insight into this as he
> re-engineer a lot of the rendering pipeline and it might be
> helpful to know what code is doing the actual text rendering. It
> used to be freetype and may still be.
> >
> > I'm also not convinced the fribidi is not the right place for
> this to happen regardless as it is responsible for taking a
> sequence of characters and generating an "displayable" sequence
> that can be rendered correctly given the correct font.
> >
> > -Steve
> >
> > On 1/23/2013 7:17 PM, Mr. Puneet Kishor wrote:
> >> Peter,
> >>
> >> My guru in all things Hindi fonts tells me that the problem
> lies at the level where encodings are converted into font
> renderings. Apparently, र् ट is a perfectly correct alternate
> rendering of the unicode, just not Correct (with a capital C). I
> have no idea where this happens with MapServer. Thankfully I've
> never had to create a map with Hindi labels (I would be curious to
> learn, perhaps off-list, who wants to see maps with Hindi labels),
> but I too would like to see this resolved.
> >>
> >> Hope someone knowledgable chimes in.
> >>
> >>
> >> On Jan 23, 2013, at 3:58 PM, "Mr. Puneet Kishor"
> <punk.kish at gmail.com <mailto:punk.kish at gmail.com>> wrote:
> >>
> >>> (adding the list back to cc so brains better than mine may help)
> >>>
> >>>
> >>> On Jan 23, 2013, at 3:47 PM, "Peter Mallen"
> <peter.mallen at airborne.aero <mailto:peter.mallen at airborne.aero>>
> wrote:
> >>>
> >>>> Hi Puneet,
> >>>>
> >>>> I have passed the codes you have provided into MapServer
> (U+0930 U+094D U+091F ) and it displays the characters
> separately, not as one character.
> >>>>
> >>>> This is how Map Server displays it: र् ट
> >>>
> >>>
> >>> hmmm... well, then this is definitely above my pay grade.
> Seems like you are doing the right thing, but my guess is
> MapServer's Unicode implementation is not working properly.
> >>>
> >>> I don't see any mention of Unicode on mapserver docs page. The
> only
> >>> potentially relevant bit I see is
> >>>
> (http://www.mapserver.org/installation/unix.html#obtaining-the-neces
> >>> s
> >>> ary-software) that libgd should be compiled against the iconv
> >>> library which provides full support for Unicode. On my MBA I get
> >>>
> >>> $ ~ > gdlib-config --libs
> >>> -lXpm -lX11 -ljpeg -lfontconfig -lfreetype -lpng -lz -liconv
> >>>
> >>>
> >>>
> >>>> Thank You,
> >>>>
> >>>> Peter Mallen
> >>>> -----Original Message-----
> >>>> From: Mr. Puneet Kishor [mailto:punk.kish at gmail.com
> <mailto:punk.kish at gmail.com>]
> >>>> Sent: Wednesday, January 23, 2013 3:36 PM
> >>>> To: Peter Mallen
> >>>> Subject: Re: [mapserver-users] Rendering Hindi Characters
> >>>>
> >>>>
> >>>> On Jan 23, 2013, at 3:00 PM, "Peter Mallen"
> <peter.mallen at airborne.aero <mailto:peter.mallen at airborne.aero>>
> wrote:
> >>>>
> >>>>> Hi Puneet,
> >>>>>
> >>>>> I did try feeding MapServer the correct Unicode characters,
> and it did fix the characters using what I believe is called the
> "Reph" symbol (ि) but I could not get the conjunct consonants to
> display correctly.
> >>>>
> >>>> I have no idea what the above means ;-), but, did you try passing
> >>>> U+0930 U+094D U+091F to MapServer? See
> >>>> http://www.unicode.org/charts/PDF/U0900.pdf
> >>>>
> >>>>
> >>>> U+0930 -> र -> r
> >>>> U+094D -> ् -> "halves" the previously typed r 091F -> ट -> thick
> >>>> U+t sound
> >>>>
> >>>>
> >>>>
> >>>>> Also, I have posted on the fribidi list and I believe it
> does only help with bidirectional languages, but I could be wrong.
> >>>>>
> >>>>> Thank You,
> >>>>>
> >>>>> Peter Mallen
> >>>>>
> >>>>> -----Original Message-----
> >>>>> From: mapserver-users-bounces at lists.osgeo.org
> <mailto:mapserver-users-bounces at lists.osgeo.org>
> >>>>> [mailto:mapserver-users-bounces at lists.osgeo.org
> <mailto:mapserver-users-bounces at lists.osgeo.org>] On Behalf Of Mr.
> >>>>> Puneet Kishor
> >>>>> Sent: Wednesday, January 23, 2013 12:27 PM
> >>>>> To: mapserver-users at lists.osgeo.org
> <mailto:mapserver-users at lists.osgeo.org>
> >>>>> Subject: Re: [mapserver-users] Rendering Hindi Characters
> >>>>>
> >>>>>
> >>>>> On Jan 23, 2013, at 12:17 PM, Stephen Woodbridge
> <woodbri at swoodbridge.com <mailto:woodbri at swoodbridge.com>> wrote:
> >>>>>
> >>>>>> On 1/23/2013 2:20 PM, Peter Mallen wrote:
> >>>>>>> Hello Everyone,
> >>>>>>>
> >>>>>>> I am having issues rendering certain Hindi characters. I am
> >>>>>>> using
> >>>>>>> UTF-8 encoding.
> >>>>>>>
> >>>>>>> When attempting to display the following Hindi character in
> >>>>>>> MapServer: र्ट
> >>>>>>>
> >>>>>>> It is rendered in MapServer as the following two characters: र्
> >>>>>>> ट
> >>>>>>>
> >>>>>>> I believe these characters form what is called a conjunct
> >>>>>>> consonant, of which there appear to be thousands of and
> most of
> >>>>>>> them do not render using MapServer.
> >>>>>>>
> >>>>>>> Also, It appears that MapServer is having an issue displaying
> >>>>>>> these conjunct consonants and wants to display these
> characters
> >>>>>>> separately rather than to form them as one combined character.
> >>>>>>>
> >>>>>>> Here is a link describing more about the Devanagari alphabet:
> >>>>>>> http://www.omniglot.com/writing/devanagari.htm
> >>>>>>>
> >>>>>>> Can anyone please help with this?
> >>>>>>>
> >>>>>>> Thank You,
> >>>>>>>
> >>>>>>> Peter Mallen
> >>>>>> Hi Peter,
> >>>>>>
> >>>>>> I can not speak to this specific problem but I can give you
> and overview. Mapserver uses fribidi library for rending this data
> and I know from working with Arabic that a lot depends on the font
> you are using, in additional to the capabilities and version of
> fribidi library.
> >>>>>
> >>>>>
> >>>>> Isn't fribidi specifically for bidirectional languages?
> Hindi is strictly LTR.
> >>>>>
> >>>>>
> >>>>>> This kind of question is probably best asked on the fribidi
> list as they will know more about what is supported or not for any
> given language and they might be able to point you to a better font.
> >>>>>>
> >>>>>> http://www.fribidi.org/
> >>>>>>
> >>>>>> You should probably make sure you are using a recent
> version of the library like 0.19.2 or better and be aware the
> 0.19.5 was released in Dec 2012.
> >>>>>>
> >>>>>
> >>>>> Peter, I forget where you and I left our off-list
> conversation re. the issues you were having with rendering
> Devnagari. Did you try feeding MapServer the correct Unicode
> characters?
> >>>>>
> >>>>> Hope you figure this out.
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Puneet Kishor
> >> _______________________________________________
> >> mapserver-users mailing list
> >> mapserver-users at lists.osgeo.org
> <mailto:mapserver-users at lists.osgeo.org>
> >> http://lists.osgeo.org/mailman/listinfo/mapserver-users
> >>
> > _______________________________________________
> > mapserver-users mailing list
> > mapserver-users at lists.osgeo.org
> <mailto:mapserver-users at lists.osgeo.org>
> > http://lists.osgeo.org/mailman/listinfo/mapserver-users
> > _______________________________________________
> > mapserver-users mailing list
> > mapserver-users at lists.osgeo.org
> <mailto:mapserver-users at lists.osgeo.org>
> > http://lists.osgeo.org/mailman/listinfo/mapserver-users
> > _______________________________________________
> > mapserver-users mailing list
> > mapserver-users at lists.osgeo.org
> <mailto:mapserver-users at lists.osgeo.org>
> > http://lists.osgeo.org/mailman/listinfo/mapserver-users
>
> _______________________________________________
> mapserver-dev mailing list
> mapserver-dev at lists.osgeo.org <mailto:mapserver-dev at lists.osgeo.org>
> http://lists.osgeo.org/mailman/listinfo/mapserver-dev
> _______________________________________________
> mapserver-users mailing list
> mapserver-users at lists.osgeo.org
> <mailto:mapserver-users at lists.osgeo.org>
> http://lists.osgeo.org/mailman/listinfo/mapserver-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapserver-dev/attachments/20130405/0942d5f6/attachment-0001.html>
More information about the mapserver-dev
mailing list