[Gdal-dev] locale handling in GDAL

[Frank Warmerdam]

Folks,

Peng Gao has pointed out that GDAL can fall over rather ungracefully if
used in a locale with odd numeric formatting rules, such as many European
locales.

For instance, in the Danish locale printf( "%g\n", 1.234) will produce
"1,234".  This causes no end of problems in all sorts of GDAL code that
depends on sprintf() producing particular formatting for writing to disk
files.  We also assume that atoi(), scanf, etc will work properly on
"normal" values such as 1.234.

In normal use, the GDAL programs are run in the C locale, and we get the
normal expected handling of numbers in text.  However, if an application
calling GDAL calls setlocale() with a non "C" locale, we can run into the
problems.  This has been encountered before in OpenEV, but I just had
OpenEV force the numeric locale to "C" since I didn't understand aspects of
the issue.   However, I can't force this non-solution on applications that
take internationalization more seriously.

I have done some experiments and found I can temporarily force the "C"
locale within GDAL itself.  I implemented this using a small class
declared in cpl_conv.h called CPLLocaleC.  When created, objects of this
class will force LC_NUMERIC to C.  When destroyed they will restore the
original locale setting.   This makes it easy and pretty safe to force
C locale within selected scopes.

I have introduced it's use into GDALOpen, GDALCreate, GDALCreateCopy and
GDALClose.  *Most* parsing and formatting for disk access takes place in
these functions and this gets things working better in many cases.  However,
it is far from comprehensive.   I could try and push this same mechanism
through the whole GDAL API, but I'm not really sure how expensive the
setlocale() call is and it would be a lot of work.

So, for now, I am asking for feedback on how others feel about this issue.

BTW, I have also introduced a --locale commandlnie option in the GDAL
standard command line option process for easier testing of locale issues.
You should be able now to do something like:

gdalinfo --debug off --locale Danish varminghede.jpg
GDAL: AutoSkipDriver(JP2KAK)
Driver: JPEG/JPEG JFIF
Size is 822, 733
Coordinate System is:
PROJCS["UTM Zone 32, Northern Hemisphere",
     GEOGCS["European 1950",
         DATUM["European 1950",
             SPHEROID["International 1909",6378388,297.0000000284045],
             TOWGS84[-87,-98,-121,0,0,0,0]],
         PRIMEM["Greenwich",0],
         UNIT["degree",0.0174532925199433]],
     PROJECTION["Transverse_Mercator"],
     PARAMETER["latitude_of_origin",0],
     PARAMETER["central_meridian",9],
     PARAMETER["scale_factor",0.9996],
     PARAMETER["false_easting",500000],
     PARAMETER["false_northing",0],
     UNIT["Meter",1]]
Origin = (-0,500000,0,500000)
Pixel Size = (1,00000000,-1,00000000)
Corner Coordinates:
Upper Left  (  -0,5000000,   0,5000000)
Lower Left  (      -0,500,    -732,500)
Upper Right (     821,500,       0,500)
Lower Right (     821,500,    -732,500)
Center      (     410,500,    -366,000)
Band 1 Block=822x1 Type=Byte, ColorInterp=Red
   Metadata:
     LAYER_TYPE=athematic
Band 2 Block=822x1 Type=Byte, ColorInterp=Green
   Metadata:
     LAYER_TYPE=athematic
Band 3 Block=822x1 Type=Byte, ColorInterp=Blue
   Metadata:
     LAYER_TYPE=athematic

Note that the mainline's reporting of coordinates with a comma instead
of the decimal.  But the WKT coordinate system is formatted in a portable
rather than locale specific fashion.

Hmm, I see the reprojection part of gdalinfo didn't work, likely
because the reprojection code was not invoked within the modified
functions and so could not parse the WKT properly.  Interesting
demonstration of how it isn't trivial to catch all important entry points.

Best regards,
-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | President OSGF, http://osgeo.org