[GRASS-dev] Re: [bug #5195] (grass) ps.map sets encoding to iso-8859-1

[Maciej Sieczka]

Glynn Clements wrote:
> Moritz Lennert wrote:

>> Most GNU/Linux distributions 
>> come with UTF-8 as default system encoding nowadays and so users will 
>> have that problem.

> The default locale's encoding doesn't matter. What matters is the
> encoding of the text in the ps.map input file.

Technically that's right, but the problem is that eg. in Ubuntu, where
default encoding is UTF-8, the new text files I create are by default
in UTF-8. Unless the user knows what the ISO-8859-whatever and
UTF-whatever are, he's in trouble. I understand we currently just have
to assume he must know that tough, and that's it's not GRASS problem
problem if he doesn't.

>> I imagine there is no way of automatically identifying the encoding of a 
>> file ?

> Correct. At least, not reliably. You can use various heuristics; e.g. 
> bytes \x80-\x9F aren't valid in any ISO-8859-* encodings, certain
> combinations aren't valid in UTF-8 etc.

ENCA [1] is a nice example of such a software. It never failed for me
for ISO-8859-1/2 <-> UTF-8 automatic conversions.

Maciek

[1] http://freshmeat.net/projects/enca/