[gdal-dev] encodings, locales ?

[Attila Csipa]

On Tuesday 19 August 2008 01.50:18 Mateusz Loskot wrote:
> Some bits of the RFC-23 implementation has been already
> submitted to the  SVN trunk. Try this queries to find detailed changesets:
>
> http://trac.osgeo.org/gdal/search?q=RFC23&noquickjump=1&changeset=on
> http://trac.osgeo.org/gdal/search?q=RFC+23&noquickjump=1&changeset=on

OK, so no driver specific changes (yet), but I don't expect those until RFC23 
is actually finished.

> > Also, is there plan for
> > doing proper localization or is it just internationalization for now ?
>
> Could you explain what you mean as "proper localization" and what is
> missing in the RFC-23 that would make it "proper"?

From what I see, RFC-23 is internationalization, which means it makes it 
possible to use it with languages other than the original (e.g. English) as 
it would get mostly language-agnostic with utf8. This is a good first step 
(with the arguably acceptable loss of information in some of the more obscure 
encodings), but it would be really good to have user-defineable input/output 
encoding, especially as not all formats define the encoding themselves (see 
dbf, csv). The output encoding problem can especially escalate if your 
original encoding was a non-latin one byte encoding. For example, if you have 
a source WIN1251 Cyrillic encoding, the utf8 version would take roughly twice 
the number of bytes, so your data would either get truncated or the field 
length must be changed to accommodate.

Localization is different as it also means functional difference in terms that 
the app actually acts differently based on your locale/encoding. In the GIS 
field this often relates to date/time format representation, but also the 
textual representation of numbers (see thousand separator, comma, as csv/xml 
imports can be especially pesky). Arguably projections could/should be part 
of this, too, but I understand that it could cause as many problems (even if 
for the right reasons) for many existing installs.

In the broadest sense this would mean the interface of the application itself 
(so, if you have a locale set, the application communicates error and help 
messages in the language and format appropriate for the given locale), but 
this is really hard to coordinate and I guess not really expected from a 
specialist tool like GDAL.