working with database in UTF-8

[Jackey Cheung]

I've checked that data in MySQL are loaded correctly with UTF-8 encoding, 
and I can select data out of it with SELECT. However, since I'm using OGR to 
connect to MySQL, I'm not sure whether there were any encoding issue here.

But I'm suspecting that the TTF font I'm using may not correct. I've tried 
some fonts, like Arial, Arial unicode, and another TTF font of Big5 
encoding. But none works. The original encoding of the text is Big5, I've 
converted and loaded them to MySQL as UTF-8, and they show up correct inside 
MySQL, what fonts should I use to display them on a map?


----- Original Message -----
From: "Stephen Woodbridge" <woodbri at SWOODBRIDGE.COM>
To: <MAPSERVER-USERS at LISTS.UMN.EDU>
Sent: Saturday, June 23, 2007 02:45
Subject: Re: [UMN_MAPSERVER-USERS] working with database in UTF-8

> Jackey,
>
> I suspect that the font file is not utf8 or the data is already mangled 
> before it gets to mapserver. The mapserver stuff works fine for me with 
> utf-8 data in shapefiles as long as I set the ENCODING 'utf8' in the 
> LABEL. A bigger issue might be:
>
> What is the encoding on the source data?
> what is the encoding on the database?
> was the data loded correct with respect to the encoding?
> if you do a select * from table in mysql does the data display correctly?
> if you slect the records with ogrinfo? to a file or you monitor do they 
> show corectly?
>
> There are a WHOLE lot oof steps where things could have broken before it 
> gets to mapserver. You need to validate the data is still good as it comes 
> into mapserver.
>
> -Steve W.
>
> Steve Lime wrote:
>> Hmmm... All the encoding parameter does is tell MapServer to convert from 
>> the supplied encoding to
>> unicode so I don't think it has any effect. If you already have 
>> unicode/utf-8 strings then this should
>> just work assuming your font has the right characters.
>>
>> Does a simple shapefile work?
>>
>> Steve
>>
>>>>> On 6/22/2007 at 2:56 AM, in message
>> <bfb704a40706220056t20220029n82b4ec43aa5e3854 at mail.gmail.com>, Jackey 
>> Cheung
>> <cheung.jackey at GMAIL.COM> wrote:
>>> Greetings!
>>>
>>> I'm trying to setup a mapserver that supports multiple languages with 
>>> MySQL.
>>> The database uses UTF-8, while the map server use a select statement to
>>> retrieve records. The layer goes as:
>>>
>>>     LAYER
>>>         NAME                        sr
>>>         STATUS                    OFF
>>>
>>>         TYPE                        POINT
>>>         CONNECTIONTYPE    OGR
>>>         CONNECTION            "<OGRVRTDataSource>
>>>                                             <OGRVRTLayer name='sr'>
>>>
>>> <SrcDataSource>MYSQL:MapSQL,user=root,password=123456,host=localhost,port=3306
>>> ,tables=POI</SrcDataSource>
>>>                                                 <SrcSQL>SELECT `Name`, 
>>> geom
>>> FROM `POI` WHERE `ID` = %o%</SrcSQL>
>>>
>>> <GeometryType>wkbPoint</GeometryType>
>>>                                             </OGRVRTLayer>
>>>                                          </OGRVRTDataSource>"
>>>         DATA                        "sr"
>>>         UNITS                        METERS
>>>
>>>         MAXFEATURES            1
>>>         LABELITEM                "Name"
>>>
>>>         CLASS
>>>             SYMBOL                    "point"
>>>             MINSIZE                    10
>>>             COLOR                        250    0    0
>>>             OUTLINECOLOR        0    128    0
>>>
>>>             LABEL
>>>                 ENCODING            "utf-8"
>>>                 COLOR                    255    0    0
>>>                 OUTLINECOLOR    255    255    255
>>>                 TYPE                    TRUETYPE
>>>                 FONT                    arial
>>>                 SIZE                    8
>>>                 ANTIALIAS            TRUE
>>>                 POSITION            UC
>>>                 PARTIALS            FALSE
>>>                 BUFFER                5
>>>                 FORCE                    TRUE
>>>             END
>>>         END
>>>     END
>>>
>>> All Chinese characters are shown as question marks (?) on the map, no 
>>> mater
>>> using arial or Chinese fonts.
>>>
>>> Any advise?