working with database in UTF-8

[Jackey Cheung]

Well, it shows a bunch of question marks (?) no matter I use arial or big5 
fonts. I don't know how to check out the encoding of fonts, and I don't know 
which font to use for the case.

If I use the shape (mapinfo) file, which is in Big5 encoding, mapserver can 
show the correct characters. I've tried converting the query result to big5 
from MySQL:
"SELECT CONVERT(name USING big5) AS name FROM table", and use big5 encoding 
and font for the layer, but it doesn't work neither, still only question 
marks.


----- Original Message -----
From: "Steve Lime" <Steve.Lime at dnr.state.mn.us>
To: "Jackey Cheung" <cheung.jackey at GMAIL.COM>
Sent: Tuesday, June 26, 2007 23:16
Subject: Re: [UMN_MAPSERVER-USERS] working with database in UTF-8

What do you see when you use a unicode Arial, are all characters garbled, a 
select few or ? If you have
a font that has big5 encoding AND also unicode that might be the ticket.

Steve


>>> On 6/26/2007 at 3:33 AM, in message 
>>> <27FD47F181414813A4276082ED03C0EC at jackey>,
Jackey Cheung <cheung.jackey at GMAIL.COM> wrote:
> I've checked that data in MySQL are loaded correctly with UTF-8 encoding,
> and I can select data out of it with SELECT. However, since I'm using OGR 
> to
>
> connect to MySQL, I'm not sure whether there were any encoding issue here.
>
> But I'm suspecting that the TTF font I'm using may not correct. I've tried
> some fonts, like Arial, Arial unicode, and another TTF font of Big5
> encoding. But none works. The original encoding of the text is Big5, I've
> converted and loaded them to MySQL as UTF-8, and they show up correct 
> inside
> MySQL, what fonts should I use to display them on a map?
>
>
> ----- Original Message -----
> From: "Stephen Woodbridge" <woodbri at SWOODBRIDGE.COM>
> To: <MAPSERVER-USERS at LISTS.UMN.EDU>
> Sent: Saturday, June 23, 2007 02:45
> Subject: Re: [UMN_MAPSERVER-USERS] working with database in UTF-8
>
>> Jackey,
>>
>> I suspect that the font file is not utf8 or the data is already mangled
>> before it gets to mapserver. The mapserver stuff works fine for me with
>> utf-8 data in shapefiles as long as I set the ENCODING 'utf8' in the
>> LABEL. A bigger issue might be:
>>
>> What is the encoding on the source data?
>> what is the encoding on the database?
>> was the data loded correct with respect to the encoding?
>> if you do a select * from table in mysql does the data display correctly?
>> if you slect the records with ogrinfo? to a file or you monitor do they
>> show corectly?
>>
>> There are a WHOLE lot oof steps where things could have broken before it
>> gets to mapserver. You need to validate the data is still good as it 
>> comes
>> into mapserver.
>>
>> -Steve W.
>>
>> Steve Lime wrote:
>>> Hmmm... All the encoding parameter does is tell MapServer to convert 
>>> from
>>> the supplied encoding to
>>> unicode so I don't think it has any effect. If you already have
>>> unicode/utf-8 strings then this should
>>> just work assuming your font has the right characters.
>>>
>>> Does a simple shapefile work?
>>>
>>> Steve
>>>
>>>>>> On 6/22/2007 at 2:56 AM, in message
>>> <bfb704a40706220056t20220029n82b4ec43aa5e3854 at mail.gmail.com>, Jackey
>>> Cheung
>>> <cheung.jackey at GMAIL.COM> wrote:
>>>> Greetings!
>>>>
>>>> I'm trying to setup a mapserver that supports multiple languages with
>>>> MySQL.
>>>> The database uses UTF-8, while the map server use a select statement to
>>>> retrieve records. The layer goes as:
>>>>
>>>>     LAYER
>>>>         NAME                        sr
>>>>         STATUS                    OFF
>>>>
>>>>         TYPE                        POINT
>>>>         CONNECTIONTYPE    OGR
>>>>         CONNECTION            "<OGRVRTDataSource>
>>>>                                             <OGRVRTLayer name='sr'>
>>>>
>>>> <SrcDataSource>MYSQL:MapSQL,user=root,password=123456,host=localhost,port=3306
>>>> ,tables=POI</SrcDataSource>
>>>>                                                 <SrcSQL>SELECT `Name`,
>>>> geom
>>>> FROM `POI` WHERE `ID` = %o%</SrcSQL>
>>>>
>>>> <GeometryType>wkbPoint</GeometryType>
>>>>                                             </OGRVRTLayer>
>>>>                                          </OGRVRTDataSource>"
>>>>         DATA                        "sr"
>>>>         UNITS                        METERS
>>>>
>>>>         MAXFEATURES            1
>>>>         LABELITEM                "Name"
>>>>
>>>>         CLASS
>>>>             SYMBOL                    "point"
>>>>             MINSIZE                    10
>>>>             COLOR                        250    0    0
>>>>             OUTLINECOLOR        0    128    0
>>>>
>>>>             LABEL
>>>>                 ENCODING            "utf-8"
>>>>                 COLOR                    255    0    0
>>>>                 OUTLINECOLOR    255    255    255
>>>>                 TYPE                    TRUETYPE
>>>>                 FONT                    arial
>>>>                 SIZE                    8
>>>>                 ANTIALIAS            TRUE
>>>>                 POSITION            UC
>>>>                 PARTIALS            FALSE
>>>>                 BUFFER                5
>>>>                 FORCE                    TRUE
>>>>             END
>>>>         END
>>>>     END
>>>>
>>>> All Chinese characters are shown as question marks (?) on the map, no
>>>> mater
>>>> using arial or Chinese fonts.
>>>>
>>>> Any advise?