[fdo-internals] SHP Provider: Writing Chinese Characters into DBF Fails When the System locale is EN.

[Dan Stoica]

Does your SHP dataset include a .cpg file? If no, you should create one. If yes, then it should contain the Chinese codepage otherwise the multibyte conversion will default to the machine's locale.

As for 11 characters limit, there is nothing you can do since this comes from the DBF specification.


From: fdo-internals-bounces at lists.osgeo.org [mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Xumeng Chen
Sent: Tuesday, May 05, 2009 5:03 AM
To: fdo-internals at lists.osgeo.org
Cc: Kenny Jian; John Jiang; Hunter Chen
Subject: [fdo-internals] SHP Provider: Writing Chinese Characters into DBF Fails When the System locale is EN.

Hi All,
Recently, I am developing with SHP Provider to export an SHP file. During testing my code, I found sometimes the attributes of the features which contain Chinese Characters can't be written into the file correctly, and my system locale is English(US), the exportation failed, because Provider can't store correct attribute name in the DBF file.

After looking into the SHP provider code, I find that when provider wrote attribute into DBF file, the wide strings will be converted into multi-bytes with the current locale codepage. For example, in my machine the Chinese strings were converted with 1252 codepage which is not correct.

After finding this, I tried to modify the source code and hardcode the codepage to UTF-8, then the characters are written and recognized correctly, and it seems that all right but there is a limitation. The max length of attributes name in DBF is 11, if we convert all string with UTF-8, only 5 letters are support when users in FR/DE locale...

Does anyone have suggestion to my situation? Like how to work around it, or suggestion to my fixing?
Any suggestion is appreciated highly.

Thanks,
Jimmy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/fdo-internals/attachments/20090505/43d7f7bf/attachment.html