[gdal-dev] How to debug the shape open option "encoding"?

[Even Rouault]

Jukka,

I believe I understood what is going on.

OSGeo4W builds do use libiconv for recoding and iconv is rather lax regarding the spelling of 
the source and target encodings. It supports at least "8859_1", "ISO-8859-1", "ISO8859-1", 
"ISO88591" and ..."ISO_8859-1" (or "LATIN1" as well)

Whereas gisinternal builds do not link against libiconv and use the GDAL builtin minimalistic 
"stub" + Windows recoding API. This interface only understands the string "ISO-8859-1", 
"UTF-8" and "CPxxx" code pages

So always use "ISO-8859-1".

I've just pushed
https://github.com/OSGeo/gdal/commit/9d3d2f715e84d9aa2ccaa72d1167a842a0bee1ed to 
have more uniformized spelling of the above.

Even

> Hi,
> 
> I need to run a certain job that requires open option "-oo
> encoding="ISO_8859-1" and while it runs fine on Windows with the OSGeo4W
> installation with version GDAL 2.2.4, released 2018/03/19 it leads to loads
> of warnings "Warning 1: Value of field 'name_field' is not a valid UTF-8
> string." with the gisinternals installation GDAL 2.3.0dev, released
> 2017/99/99.
> 
> The shapefiles which I have are really using ISO_8859-1 and gisinternals
> version handles them OK without using the open option, except one. The
> problematic one does not have the LDID "87 / 0x57" flag in the .dbf part
> and therefore it is interpreted as UTF-8 if I do not use the open option.
> If I use the open option the gisinternals version prints the warnings about
> all the Shape: Treating as encoding 'ISO_8859-1'.
> Shape: Cannot recode from 'ISO_8859-1'. Disabling recoding
> 
> Is there anything else that an end user could do for finding out why the
> gisinternals build fails with recoding? BTW is it documented somewhere
> which are the correct values for different encodings? For example in my
> case I had to use exactly "-oo encoding="ISO_8859-1".
> 
> -Jukka Rahkonen-


-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20180511/c954f9e1/attachment.html>