[fdo-users] FDO OGR 3.6+3.7 and UTF-8 problem

[Frank Warmerdam]

On 12-01-06 04:57 AM, Hans Milling wrote:
> Hi everyone
>
> I need some help. I have problems with none ascii characters with FDO OGR
> and MGOS2.2.
> The strings on the map like road names are all messed up if they contain a
> danish letter like Æ Ø Å.
> A city name like "Farsø" is suddently written "Fars؀".
> I have created a small test program (see code below) to test the problem,
> and FDO 3.3 does not have any issues, but FDO 3.6 and 3.7 seems to have this
> issue. To me the ISO-8859-1 string read from the TAB file is converted to
> UTF-8 at some point, and that messes up the text. See this image for the
> output:
> http://osgeo-org.1803224.n2.nabble.com/file/n7158330/FDO.png
> Road name: "Bakkegårdsvej", the å character (number 197 or 0xc5) is treated
> as unicode (3 bytes) and thus the following "rd" letters are included to
> create a Chinese character resulting in "Bakkeg岤svej".
> Does anyone have a fix for this, can I recompile FDO ine some way to not
> make this error?
> I think FDO should know/detect the format of the strings from the source, so
> that these are not destroyed.

Hans,

The relevant RFC in OGR is:

   http://trac.osgeo.org/gdal/wiki/rfc23_ogr_unicode

It appears the FDO OGR provider should at the very least be checking
the OLCStringsasUTF8 capability on the layer.  If true it should be
assumed string attributes from the layer are in UTF8 and processed
accordingly.

Best regards,
-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/warmerda
and watch the world go round - Rush    | Geospatial Software Developer