[mapserver-users] Confirmation of status of UTF8 support, and where transcoding to Latin-1 may be happening.
Russell McOrmond
russell at flora.ca
Thu Feb 5 08:00:46 PST 2009
This will likely be my final update on this specific, given I believe
I've found all our issue.
The problem came down to two unrelated problems, eithor of which meant
that nationally encoded strings from ArcSDE weren't being properly handled
in mapserver.
a) gd 2.0.35
This version of GD was incorrectly detecting the iconv library due to a
missing macro.
configure.ac:64: error: possibly undefined macro: AM_ICONV
This meant that it was missing the '#define HAVE_ICONV 1' in the
config.h.
If this is not defined, this activates a set of stub functions in
gdkanji.c . This was the source of iconv_open() always returning -1.
This problem is fixed in gd 2.0.36RC1 . There is likely a way to add
the right macro with 2.0.35 and use it as well.
b) endianness setting with NSTRING data from SDE
See http://trac.osgeo.org/mapserver/ticket/2878
http://lists.gnu.org/archive/html/bug-gnu-libiconv/2009-02/msg00001.html
It turns out that we need to specify the endianness for the NSTRING data
(UTF-16 encoded) coming from libsde. The glibc version of the iconv()
functions seem to assume little-endian for "UTF-16", while the libiconv
version seems to assume network order or big-endian.
If we specify "UTF-16LE", it will always work regardless of which
iconv() version we are using.
With this problem now solved, I just wish the problem with zlib in
libsde was resolved.
http://forums.esri.com/Thread.asp?c=2&f=1718&t=212867
If libsde from ArcSDE 9.3 is still using a known insecure version of
zlib from 1998, it is quite likely that other aspects of ESRI tools are
using this as well. I am surprised to learn that this was not considered
a high priority for ESRI, given the most recent version of zlib is from
2005 and there has been plenty of opportunity for distributing a fix.
--
Russell McOrmond, Internet Consultant: <http://www.flora.ca/>
Please help us tell the Canadian Parliament to protect our property
rights as owners of Information Technology. Sign the petition!
http://digital-copyright.ca/petition/ict/ http://KillBillC61.ca
"The government, lobbied by legacy copyright holders and hardware
manufacturers, can pry control over my camcorder, computer,
home theatre, or portable media player from my cold dead hands!"
More information about the MapServer-users
mailing list