[GRASS-dev] wxgui encoding issues

[Glynn Clements]

Moritz Lennert wrote:

> Using any version, I have the following encoding issues in the GUI 
> (knowing that my machine is configured with locale fr_BE.UTF-8) :
> 
> - r.reclass interactive input with accents gives me
> 
> "Traceback (most recent call last):
>    File 
> "/home/mlennert/SRC/GRASS/grass_trunk/dist.i686-pc-linux-gnu/etc/gui/wxpython/gui_core/forms.py", 
> line 1738, in OnFileText
>      f.write(text)
> UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in 
> position 7: ordinal not in range(128)"

File objects will normally[1] use ASCII as the encoding, regardless of
any environment settings. Writing a unicode object to a file will
convert it to a byte string according the file's encoding (or, if it
doesn't have one, according the default encoding, which is normally
ASCII).

[1] One exception is that a tty will automatically use the
environment's encoding.

> The same data in a file works without issues.

If you tell r.reclass to read from a file, Python doesn't get
involved, and the data stays as bytes.

Data entered via wxWidgets will originate as Unicode. The
environment's encoding only matters if the Python code explicitly 
makes it matter.

Python's built-in conversions between "str" and "unicode" types use
the default encoding, which is independent of the environment (it can
be set in sitecustomize.py, but cannot be changed after start-up).

If you want to read/write Unicode data from/to files (including pipes,
etc), you either need to use codecs.open() instead of the built-in
open() or file(), or explicitly use the str.decode() and/or
unicode.encode() methods.

Most GRASS libraries and modules don't care about the encoding; it's
all just bytes. If you use non-ASCII characters in a reclass file,
categories, etc, the file may not be interpreted correctly on another
system with a different encoding.

-- 
Glynn Clements <glynn at gclements.plus.com>