[GRASS-dev] [GRASS GIS] #3639: Write table like output directly to database

[GRASS GIS]

#3639: Write table like output directly to database
--------------------------+----------------------------------
  Reporter:  sbl          |      Owner:  grass-dev@…
      Type:  enhancement  |     Status:  new
  Priority:  normal       |  Milestone:  8.0.0
 Component:  Database     |    Version:  svn-trunk
Resolution:               |   Keywords:  table,database,write
       CPU:  Unspecified  |   Platform:  Unspecified
--------------------------+----------------------------------

Comment (by sbl):

 I am considering adding a function to the Python scripting library for
 parsing table-like stdout from GRASS modules into Numpy arrays. Nothing
 complicated, but probably a convenient wrapper function!?

 See:

 {{{
 def stdout2numpy(stdout=None, sep=',', names=False, null_value=None,
                  fill_value=None, comments='#', usecols=None):
     """Read table-like output from grass modules as Numpy array;
     format instructions are handed down to Numpys genfromtxt function

     param str|byte stdout: tabular stdout from GRASS GIS module call
     param str sep: Separator delimiting columns
     param list names: List of strings with names for columns
     param str null_value: Characters representing the no-data value
     param str fill_value: Value to fill no-data with
     param str comments: Character that
     param list usecols: List of columns to import

     """
     import numpy as np
     from io import BytesIO
     if type(stdout) == str:
         stdout = gscript.encode(stdout)
     elif type(stdout) != byte:
         gscript.fatal(_('Unsupported data type'))
     np_array = np.genfromtxt(BytesIO(stdout),
                              missing_values=null_value,
                              filling_values=fill_value,
                              usecols=usecols,
                              names=names,
                              dtype=None, delimiter=sep)
     return np_array

 }}}

 or alternatively as np_parse_command (equivalent to parse_command()):

 {{{
 def np_parse_command(*args, **kwargs):
     """Passes all arguments to read_command, then parses the output
     using Numpys genfromtxt() function to generate a Numpy array from
     table like output.

     Parsing options in Numpys genfromtxt() function can be
     optionally given by <em>parse</em> dictionary, e.g.

     ::

         parse_command(..., parse = { 'delimiter' : '|' }))

     As far as possible, at least standard parser options in GRASS
     commands are handed down and translated to genfromtxt()
     automatically, e.g. separator (GRASS) -> delimiter (Numpy).

     import numpy as np
     from io import BytesIO
     stdout = read_command(*args, **kwargs)
     if type(stdout) == str:
         stdout = gscript.encode(stdout)
     elif type(stdout) != byte:
         gscript.fatal(_('Unsupported data type'))


     filling_values = parse['filling_values'] if 'filling_values' in parse
 else np.nan
     usecols = parse['usecols'] if 'usecols' in parse else None
     names = parse['names'] if 'names' in parse else None
     dtype = parse['dtype'] if 'dtype' in parse else None

     # Check if separator is specified in GRASS command
     if 'delimiter' not in parse:
         if 'separator' in args:
             parse['delimiter'] = args['separator']
         elif 'separator' in kwargs:
             parse['delimiter'] = kwargs['separator']
         else:
             parse['delimiter'] = ','

     # Check if null_value is specified in GRASS command
     if 'missing_values' not in parse:
         if 'null_value' in args:
             parse['missing_values'] = args['null_value']
         elif 'null_value' in kwargs:
             parse['missing_values'] = kwargs['null_value']
         else:
             parse['missing_values'] = '*'

     np_array = np.genfromtxt(BytesIO(stdout), **parse)
     return np_array

 }}}

 That would probably be a first step for a Python function that can write
 output to DB...

 Any thoughts/opinions?

-- 
Ticket URL: <https://trac.osgeo.org/grass/ticket/3639#comment:2>
GRASS GIS <https://grass.osgeo.org>