[GRASS-user] WinGRASS not recognizing accented utf-8 (nor cp1252) attribute values

[Helmut Kudrnovsky]

Salim Razzaz wrote
> Hi,
> 
> I've got shape files with Swedish accented letters (ÄÖÅ) in the some of
> the attribute
> values. The Attributes are shwon as they should in the GUI. SQL
> statements, however,
> are not recognizing them. They're also messed up in the command output if
> another
> (not accented) values are queried.
> 
> I sat GRASS_DB_ENCODING to cp1252 firstly and it didn't work. Then I
> converted the
> dbf file into utf-8 and sat it as the value of the variable, to no avail.
> I also
> tried using the 'encoding' parameter in v.in.ogr in both cases, didn't
> work.
> 
> I tried it on windows 8.1 and windows 10. The same is happening in both,
> stable GRASS
> 7.0.5 and GRASS 7.2.0RC1.
> 
> The problem is only happening on Windows. Fedora and Mac OsX don't have
> this issue
> with the same shape files.
> 
> Any suggestion?
> 
> Salim

confirmed with

GRASS version: 7.3.svn                                                          
GRASS SVN revision: r70001                                                      
Build date: 2016-12-06                                                          
Build platform: x86_64-w64-mingw32                                              
GDAL: 2.1.2                                                                     
PROJ.4: 4.9.3                                                                   
GEOS: 3.5.0                                                                     
SQLite: 3.14.1                                                                  
Python: 2.7.5                                                                   
wxPython: 2.8.12.1                                                              
Platform: Windows-8-6.2.9200 (OSGeo4W) 

and a test vector with following attributes

v.db.select map=test_points at data file=D:\temp\test_point.txt

cat|id|names
1|1|ÄÖÅ
2||Æ
3||Ø
4||Å,å,Æ,æ,Ø,ø
5||ø, Ø
6||Þ
7||Ð
8||Å
9||æ

e.g.

d.vect map=test_points2 at data where="names = 'Å,å,Æ,æ,Ø,ø'" width=1
icon=basic/point size=10

doesn't show the selected point in the map display.

e.g.

v.report map=test_points at data option=coor                                       
cat|id|names|x|y|z
1|1|ÄÖÅ|1.37409120951759|47.039352838731|0.0
2||Æ|2.62326503635168|28.5515802015863|0.0
3||Ø|44.095836087244|57.2825782187707|0.0
4||Å,å,Æ,æ,Ø,ø|30.8545935228025|49.787535257766|0.0
5||ø, Ø|10.1183079973563|51.0367090846001|0.0
6||Þ|20.361533377396|52.0360481460674|0.0
8||Ã…|15.1491119517375|60.3621017805262|0.0
9||æ|-1.26290587954035|52.5879880709736|0.0

Traceback (most recent call last):
  File "C:\OSGEO4~1\apps\grass\grass-7.3.svn\gui\wxpython\gu
i_core\goutput.py", line 473, in OnCmdOutput

self.cmdOutput.AddStyledMessage(message, type)
  File "C:\OSGEO4~1\apps\grass\grass-7.3.svn\gui\wxpython\gu
i_core\goutput.py", line 772, in AddStyledMessage

self.AddTextWrapped(message, wrap=None)
  File "C:\OSGEO4~1\apps\grass\grass-7.3.svn\gui\wxpython\gu
i_core\goutput.py", line 721, in AddTextWrapped

txt = EncodeString(txt)
  File "C:\OSGEO4~1\apps\grass\grass-7.3.svn\gui\wxpython\co
re\gcmd.py", line 97, in EncodeString

return string.encode(_enc)
  File "C:\OSGEO4~1\apps\Python27\lib\encodings\cp1252.py",
line 12, in encode

return codecs.charmap_encode(input,errors,encoding_table)
UnicodeDecodeError
:
'ascii' codec can't decode byte 0xc3 in position 3: ordinal
not in range(128)

please open a ticket in trac (https://trac.osgeo.org/grass) with some
example data and an example workflow which can the devs test.






-----
best regards
Helmut
--
View this message in context: http://osgeo-org.1560.x6.nabble.com/WinGRASS-not-recognizing-accented-utf-8-nor-cp1252-attribute-values-tp5298708p5298806.html
Sent from the Grass - Users mailing list archive at Nabble.com.