[mapserver-users] UTF-8 right to left text

[Stephen Woodbridge]

On 2/27/2012 12:03 AM, Ian Walberg wrote:
> Steve,
>
> Many thanks for getting back to me.
>
> We have built with that version of fridibi and do get some text labels
> displaying correctly.
>
> The source data is from an excel spread sheet which we export to CSV and
> import into the SQLite DB.

In general, I believe that Excel uses UTF-16 or UTF-32 or UNICODE (can't 
remember witch), but if you move the csv file to Linux and do a "file 
myfile.csv" it will tell you. You can then use iconv to convert the file 
to UTF8. Alternatively, you might be able to specify the on the LABEL 
ENCODING "UTF16" or whatever it is.

I am not sure how SQLite handles character encoding or if it does more 
than just store the data you import with or with changing the encoding.

> The plan tomorrow is to find a couple of good and bad labels and follow
> these through to see if we can find where there are differences, I
> looked last week at the hex codes for a good and bad name but did not
> come to any clear confusion.
>
> Does fribidi need  RTL code to display the text correctly? Do you know
> how to decode that from the hex?

fribidi just knows how to order the RTL or LTR data. If you mix arabic 
and english in the same string, it will display them both correctly.

Regards,
   -Steve

> Regards
>
> Ian
>
> -----Original Message-----
> From: mapserver-users-bounces at lists.osgeo.org
> [mailto:mapserver-users-bounces at lists.osgeo.org] On Behalf Of Stephen
> Woodbridge
> Sent: Sunday, February 26, 2012 8:52 PM
> To: mapserver-users at lists.osgeo.org
> Subject: Re: [mapserver-users] UTF-8 right to left text
>
> On 2/26/2012 10:39 PM, Ian Walberg wrote:
>> Hello lists,
>>
>> We cannot get all our Arabic text labels to display correctly, it
>> appears that the incorrectly rendered text is not being drawn right to
>
>> left. Some work and others do not.
>>
>> Our assumption is that there are some non displaying codes we are
>> missing but we do not know enough yet to work out if this is the case.
>>
>> The data is in an sqlite database using an OGR connection.
>>
>> Any ideas or experience that you can share would be fabulous.
>
> Ian,
>
> You need the follow pieces to get this to work:
>
> 1. make sure you have built mapserver with fribidi-0.19.2 2. you need a
> ttf that supports the character set you are using
>      - I always make sure my data is in UTF8
>      - and I have a unicode arial.ttf font from Windows 3. make sure your
> LABEL block has ENCODING "UTF8" in it
>      - assuming your data is in UTF8
>
> On Linux you can do:
> $ dbfdump myshapefile.dbf>  aaa
> $ file aaa
>
> and it will tell you what the encoding is.
>
> Issues that I have run into are:
>
> 1. data with mixed utf8 and other encodings like Window-1256 or
> ISO-8859-6 2. In UTF8 data there are a lot of glyphs that are connecting
> characters or non printing characters and many fonts do not have glyphs
> defined for these characters which typically causes a square box to be
> displayed.
>
> -Steve W
> _______________________________________________
> mapserver-users mailing list
> mapserver-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapserver-users