[Qgis-user] Problem with processing and GRASS algorithms in QGIS 3

[Stephen]

Hi everyone,

I'm porting some PyQGIS 2 code to 3.

This post is a continuation of the issue I first mentioned in this post
to the list:

http://lists.osgeo.org/pipermail/qgis-user/2018-May/042343.html

I've since upgraded from 3.0.1 to 3.0.2. (Previously working code did
not work with 3.1 master.)

My code contains the following statement:
> grid_r_raster_layer = processing.run("grass7:v.to.rast", {'input': grid_v, 'type': 0, 'use': 1,
>                                     'GRASS_REGION_PARAMETER': extent,
>                                     'output': file_grid_r,
>                                     'GRASS_REGION_CELLSIZE_PARAMETER': 0.008333,
>                                     'GRASS_SNAP_TOLERANCE_PARAMETER': -1,
>                                     'GRASS_MIN_AREA_PARAMETER': 0.000100},
>                              feedback=QgsProcessingFeedback())['output']

'grid_v' is a QgsVectorLayer object and 'file_grid_r' a string
containing a path:

>>>> type(grid_v)
> <class 'qgis._core.QgsVectorLayer'>
>>>> grid_v.isValid()
> True
>>>> print(file_grid_r)
> /tmp/processing_3951f631a6664dd2ac1924f0536a7dd0/5b088658e2ff22

If I execute this code in the pycharm debugger, I get this output:

> Traceback (most recent call last):
>   File "/usr/share/qgis/python/plugins/processing/algs/grass7/Grass7Algorithm.py", line 384, in processAlgorithm
>     Grass7Utils.executeGrass(self.commands, feedback, self.outputCommands)
>   File "/usr/share/qgis/python/plugins/processing/algs/grass7/Grass7Utils.py", line 367, in executeGrass
>     startupinfo=si if isWindows() else None
>   File "/usr/lib64/python3.5/subprocess.py", line 676, in __init__
>     restore_signals, start_new_session)
>   File "/usr/lib64/python3.5/subprocess.py", line 1211, in _execute_child
>     executable = os.fsencode(executable)
>   File "/usr/lib64/python3.5/os.py", line 864, in fsencode
>     raise TypeError("expect bytes or str, not %s" % type(filename).__name__)
> TypeError: expect bytes or str, not NoneType

(This is progress! In 3.0.1, I didn't get any traceback at all.)

The problem would appear to be in Grass7Utils.py (or be triggered there;
perhaps my input is faulty). Here's the code around line 367 (first line
is 359, last is 379)

>         with subprocess.Popen(
>                 command,
>                 shell=True if isMac() else False,
>                 stdout=subprocess.PIPE,
>                 stdin=subprocess.DEVNULL,
>                 stderr=subprocess.STDOUT,
>                 universal_newlines=True,
>                 env=grassenv,
>                 startupinfo=si if isWindows() else None
>         ) as proc:
>             for line in iter(proc.stdout.readline, ''):
>                 if 'GRASS_INFO_PERCENT' in line:
>                     try:
>                         feedback.setProgress(int(line[len('GRASS_INFO_PERCENT') + 2:]))
>                     except:
>                         pass
>                 else:
>                     if 'r.out' in line or 'v.out' in line:
>                         grassOutDone = True
>                     loglines.append(line)
>                     feedback.pushConsoleInfo(line)

The offending statement in subprocess.py is this:

>             self._execute_child(args, executable, preexec_fn, close_fds,
>                                 pass_fds, cwd, env,
>                                 startupinfo, creationflags, shell,
>                                 p2cread, p2cwrite,
>                                 c2pread, c2pwrite,
>                                 errread, errwrite,
>                                 restore_signals, start_new_session)

('executable' is None in this case, which seems wrong, but I have no
clue where it comes from.)

The problem is not, as far as I can tell, in GRASS. The batch job gets
generated and when I run it manually, it works fine:

> /tmp/processing_b02dc280166f49e0af77208296099ea5/grassdata $ export GRASS_BATCH_JOB="$PWD/grass_batch_job.sh"
> /tmp/processing_b02dc280166f49e0af77208296099ea5/grassdata $ grass72
> Cleaning up temporary files...
> Starting GRASS GIS...
> Executing </tmp/processing_b02dc280166f49e0af77208296099ea5/grassdata/grass_batch_job.sh> ...
> Projection information updated
> Over-riding projection check
> Check if OGR layer <Kenya_power_grid> contains polygons...
>  100%
> WARNING: Writing column <BANDWIDTH> with integer 64 as integer 32
> WARNING: Writing column <AICD> with integer 64 as integer 32
> Importing 48 features (OGR layer <Kenya_power_grid>)...
>  100%
> -----------------------------------------------------
> Building topology for vector map <vector_5b086ca96575b3 at PERMANENT>...
> Registering primitives...
> 61 primitives registered
> 180 vertices registered
> Building areas...
>  100%
> 0 areas built
> 0 isles built
> Attaching islands...
> Attaching centroids...
>  100%
> Number of nodes: 63
> Number of primitives: 61
> Number of points: 0
> Number of lines: 61
> Number of boundaries: 0
> Number of centroids: 0
> Number of areas: 0
> Number of isles: 0
> Writing raster map...
>  100%
> v.to.rast complete.
> Checking GDAL data type and nodata value...
>  100%
> Using GDAL data type <Int32>
> Input raster map contains cells with NULL-value (no-data). The value
> -2147483648 will be used to represent no-data values in the input map. You
> can specify a nodata value with the nodata option.
> Exporting raster data to GTiff format...
> ERROR 6: SetColorTable() only supported for Byte or UInt16 bands in TIFF format.
>  100%
> r.out.gdal complete. File
> </tmp/processing_b02dc280166f49e0af77208296099ea5/5b086c8c419362> created.
> Execution of </tmp/processing_b02dc280166f49e0af77208296099ea5/grassdata/grass_batch_job.sh> finished.
> Cleaning up temporary files...

What can I do to determine the cause and make this work?

Cheers

Stephen