[GRASS-dev] [GRASS GIS] #2617: wxgui Raster query redirect to console UnicodeDecodeError

[GRASS GIS]

#2617: wxgui Raster query redirect to console UnicodeDecodeError
-----------------------------+----------------------------------------------
 Reporter:  marisn           |       Owner:  grass-dev@…              
     Type:  defect           |      Status:  new                      
 Priority:  normal           |   Milestone:  7.0.1                    
Component:  wxGUI            |     Version:  svn-trunk                
 Keywords:  query, encoding  |    Platform:  MSWindows Vista          
      Cpu:  Unspecified      |  
-----------------------------+----------------------------------------------

Comment(by marisn):

 No, this is not a solution - it is still broken.

 I appended a following print in query.py L185:
 {{{
 print 'k: %s (%s) v: %s (%s)' % (k, type(k), v, type(v))
 }}}

 And here is output:
 {{{
 k: east, north (<type 'unicode'>) v: 622578.672986, 6399325.43444 (<type
 'str'>)
 k: dores_idw at kalistrats (<type 'unicode'>) v: {'nosaukums': '',
 'kr\xc4\x81sa': '255:202:000', 'v\xc4\x93rt\xc4\xabba':
 '71.6390742964988'} (<type 'dict'>)
 k: nosaukums (<type 'str'>) v:  (<type 'str'>)
 k: krāsa (<type 'str'>) v: 255:202:000 (<type 'str'>)
 Traceback (most recent call last):
   File "C:\Program Files\GRASS GIS
 7.1.svn\gui\wxpython\mapwin\buffered.py", line 1230, in
 MouseActions

 self.OnLeftUp(event)
   File "C:\Program Files\GRASS GIS
 7.1.svn\gui\wxpython\mapwin\buffered.py", line 1407, in
 OnLeftUp

 self.mapQueried.emit(x=self.mouse['end'][0],
 y=self.mouse['end'][1])
   File "C:\Program Files\GRASS GIS
 7.1.svn\etc\python\grass\pydispatch\signal.py", line 229, in
 emit

 dispatcher.send(signal=self, *args, **kwargs)
   File "C:\Program Files\GRASS GIS
 7.1.svn\etc\python\grass\pydispatch\dispatcher.py", line
 349, in send

 **named
   File "C:\Program Files\GRASS GIS
 7.1.svn\etc\python\grass\pydispatch\robustapply.py", line
 60, in robustApply

 return receiver(*arguments, **named)
   File "C:\Program Files\GRASS GIS
 7.1.svn\gui\wxpython\mapdisp\frame.py", line 868, in Query

 self.QueryMap(east, north, qdist, rast, vect)
   File "C:\Program Files\GRASS GIS
 7.1.svn\gui\wxpython\mapdisp\frame.py", line 922, in
 QueryMap

 self.dialogs['query'] = QueryDialog(parent = self, data =
 result)
   File "C:\Program Files\GRASS GIS
 7.1.svn\gui\wxpython\gui_core\query.py", line 46, in
 __init__

 self._model = QueryTreeBuilder(self.data,
 column=self._colNames[1])
   File "C:\Program Files\GRASS GIS
 7.1.svn\gui\wxpython\gui_core\query.py", line 201, in
 QueryTreeBuilder

 addNode(parent=model.root, data=part, model=model)
   File "C:\Program Files\GRASS GIS
 7.1.svn\gui\wxpython\gui_core\query.py", line 190, in
 addNode

 addNode(parent=node, data=v, model=model)
   File "C:\Program Files\GRASS GIS
 7.1.svn\gui\wxpython\gui_core\query.py", line 187, in
 addNode

 k = DecodeString(k)
   File "C:\Program Files\GRASS GIS
 7.1.svn\gui\wxpython\core\gcmd.py", line 76, in DecodeString

 return string.decode(_enc)
   File "C:\Program Files\GRASS GIS
 7.1.svn\Python27\lib\encodings\cp1257.py", line 15, in
 decode

 return codecs.charmap_decode(input,errors,decoding_table)
 UnicodeDecodeError
 :
 'charmap' codec can't decode byte 0x81 in position 3:
 character maps to <undefined>
 }}}
 As it is visible from the output, on the second line v contains UTF-8
 encoded text. Lines 3 and 4 report it to be a str and thus a DecodeString
 is called. So far - nothing bad, but there kicks in DecodeString - it is
 using GetSystemEncoding to decode string. On this system _enc variable is
 set to cp1257 - this is definitely not UTF-8 and thus decoding fails.
 The string in question (krāsa) is coming form the GRASS translation to
 Latvian language - to reproduce the issue on your system, you must
 translate "color" to a word with non-ascii letters in it (zbarvenã) and,
 of course, encode translation file (PO) as UTF-8.

 The source of problem is r47310 where instead of installing unicode
 version of gettext a bytestring version is installed. This should work
 fine, but now in every place where a _() call is made, it returns str for
 unicode translations. Reverting r47310 fixes this bug (and probably others
 too!) without any problems, still I would like to hear Glynn's rationale
 why it was necessary in the first place (preferably with patches that
 solve _() issue if r47310 is to stay). Not using unicode version of
 gettext is really strange, as Slovenian is the only language NOT using
 UTF-8 in their PO files and it has seen the last update in 2005, thus
 GRASS PO files ARE unicode-ready.

-- 
Ticket URL: <http://trac.osgeo.org/grass/ticket/2617#comment:2>
GRASS GIS <http://grass.osgeo.org>