[GRASS-dev] [GRASS GIS] #3392: t.register: encoding error

[GRASS GIS]

#3392: t.register: encoding error
--------------------------+---------------------------------
  Reporter:  mlennert     |      Owner:  grass-dev@…
      Type:  defect       |     Status:  new
  Priority:  normal       |  Milestone:  7.2.2
 Component:  Temporal     |    Version:  svn-trunk
Resolution:               |   Keywords:  t.register encoding
       CPU:  Unspecified  |   Platform:  Unspecified
--------------------------+---------------------------------

Comment (by mlennert):

 Replying to [comment:12 annakrat]:
 > The problem mostly comes from calling gisenv function from script.utils
 which returns unicode.

 So you are saying that this has nothing to do with translated message
 strings ?

 >This was done some time ago to make the library Python3 compatible. So we
 eventually need to move >towards unicode, but to have a quick fix for the
 release, we can keep the temporal library using >bytestrings. This is a
 possible solution:
 >
 >
 > {{{
 > Index: lib/python/temporal/core.py
 > ===================================================================
 > --- lib/python/temporal/core.py       (revision 71395)
 > +++ lib/python/temporal/core.py       (working copy)
 > @@ -542,9 +542,9 @@
 >      grassenv = gscript.gisenv()
 >
 >      # Set the global variable for faster access
 > -    current_mapset = grassenv["MAPSET"]
 > -    current_location = grassenv["LOCATION_NAME"]
 > -    current_gisdbase = grassenv["GISDBASE"]
 > +    current_mapset = gscript.encode(grassenv["MAPSET"])
 > +    current_location = gscript.encode(grassenv["LOCATION_NAME"])
 > +    current_gisdbase = gscript.encode(grassenv["GISDBASE"])
 >
 >      # Check environment variable GRASS_TGIS_RAISE_ON_ERROR
 >      if os.getenv("GRASS_TGIS_RAISE_ON_ERROR") == "True" or \
 > @@ -1107,6 +1107,7 @@
 >                          detect_types=self.dbmi.PARSE_DECLTYPES |
 self.dbmi.PARSE_COLNAMES)
 >                  self.connection.row_factory = self.dbmi.Row
 >                  self.connection.isolation_level = None
 > +                self.connection.text_factory = str
 >                  self.cursor = self.connection.cursor()
 >                  self.cursor.execute("PRAGMA synchronous = OFF")
 >                  self.cursor.execute("PRAGMA journal_mode = MEMORY")
 >
 > }}}
 >

 Just the line


 {{{
 self.connection.text_factory = str
 }}}

 already suffices to solve the above-mentioned issues (and I've gone a bit
 further in the tutorial and tested a series of other t.* modules, but
 haven't encountered any other error) but it's better to also protect
 against non-ascii gisdbase variables which wasn't an issue here for me.

 Thanks Anna !!

 I have to admit that I have no idea how this works. Some form of
 documentation of the entire system would be very helpful, in line with
 what Pietro said.

 > plus all gisenv calls in temporal modules need to be encoded to not let
 any unicode into the library. I would appreciate if somebody could test it
 more thoroughly. This would be a temporary solution, we need to move to
 unicode eventually, but I think that would require more time to fix and
 more significant changes.

 Can we at least already commit this small change which already makes a big
 difference ?

 > Also, this works only for sqlite3 backend, I haven't looked at
 postgres...

 How does the database backend come in when dealing with these strings ?

 Moritz

--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3392#comment:14>
GRASS GIS <https://grass.osgeo.org>