[Qgis-user] Problem with processing and GRASS algorithms in QGIS 3
Stephen
posting at vodacomm.ca
Sat May 26 05:22:12 PDT 2018
Hi again,
I've made some more progress on this.
The problem starts in line 350 in the executeGrass() method of
Grass7Utils.py
https://github.com/qgis/QGIS/blob/307d082e3de804064a7295aa079ee6cd0e47845a/python/plugins/processing/algs/grass7/Grass7Utils.py#L350
> @staticmethod
> def executeGrass(commands, feedback, outputCommands=None):
> loglines = []
> loglines.append(Grass7Utils.tr('GRASS GIS 7 execution console output'))
> grassOutDone = False
> command, grassenv = Grass7Utils.prepareGrassExecution(commands)
> #QgsMessageLog.logMessage('exec: {}'.format(command), 'DEBUG', Qgis.Info)
'command' is a Python list; When prepareGrassExecution() returns
command, the first element is empty.
So what is happening here? Let's look in prepareGrassExecution() (line
339) to see how command is prepared:
https://github.com/qgis/QGIS/blob/307d082e3de804064a7295aa079ee6cd0e47845a/python/plugins/processing/algs/grass7/Grass7Utils.py#L339
> command = [Grass7Utils.command,
> os.path.join(Grass7Utils.grassMapsetFolder(), 'PERMANENT'),
> '--exec', Grass7Utils.grassBatchJobFilename()]
The attribute Grass7Utils.command is not set. It is set by default to
None in line 73 and then not after that.
Let me stop execution before line 339 in the debugger and do some
inspection in the console:
>>>> Grass7Utils.command
>>>> type(Grass7Utils.command)
> <class 'NoneType'>
The method Grass7Utils.grassBin() finds the path to the binary:
>>>> Grass7Utils.grassBin()
> '/usr/bin/grass72'
And now, command is properly set:
>>>> type(Grass7Utils.command)
> <class 'str'>
>>>> Grass7Utils.command
> '/usr/bin/grass72'
Anyway, my code then runs as expected after this point.
I think this is a bug no matter how one looks at it. If a path variable
is not being set even though GRASS is installed and working, that's a
bug. If it only occurs because there is something faulty about my input,
then it's still a bug because it fails to validate the input and give an
appropriate error message.
To me, the obvious fix is to call grassBin() once before line 339, but
maybe that would cause other problems, I don't know.
I'd be grateful for any suggestions.
Cheers
Stephen
On 26.05.2018 00:18, Stephen wrote:
> Hi everyone,
>
> I'm porting some PyQGIS 2 code to 3.
>
> This post is a continuation of the issue I first mentioned in this post
> to the list:
>
> http://lists.osgeo.org/pipermail/qgis-user/2018-May/042343.html
>
> I've since upgraded from 3.0.1 to 3.0.2. (Previously working code did
> not work with 3.1 master.)
>
> My code contains the following statement:
>> grid_r_raster_layer = processing.run("grass7:v.to.rast", {'input': grid_v, 'type': 0, 'use': 1,
>> 'GRASS_REGION_PARAMETER': extent,
>> 'output': file_grid_r,
>> 'GRASS_REGION_CELLSIZE_PARAMETER': 0.008333,
>> 'GRASS_SNAP_TOLERANCE_PARAMETER': -1,
>> 'GRASS_MIN_AREA_PARAMETER': 0.000100},
>> feedback=QgsProcessingFeedback())['output']
>
> 'grid_v' is a QgsVectorLayer object and 'file_grid_r' a string
> containing a path:
>
>>>>> type(grid_v)
>> <class 'qgis._core.QgsVectorLayer'>
>>>>> grid_v.isValid()
>> True
>>>>> print(file_grid_r)
>> /tmp/processing_3951f631a6664dd2ac1924f0536a7dd0/5b088658e2ff22
>
> If I execute this code in the pycharm debugger, I get this output:
>
>> Traceback (most recent call last):
>> File "/usr/share/qgis/python/plugins/processing/algs/grass7/Grass7Algorithm.py", line 384, in processAlgorithm
>> Grass7Utils.executeGrass(self.commands, feedback, self.outputCommands)
>> File "/usr/share/qgis/python/plugins/processing/algs/grass7/Grass7Utils.py", line 367, in executeGrass
>> startupinfo=si if isWindows() else None
>> File "/usr/lib64/python3.5/subprocess.py", line 676, in __init__
>> restore_signals, start_new_session)
>> File "/usr/lib64/python3.5/subprocess.py", line 1211, in _execute_child
>> executable = os.fsencode(executable)
>> File "/usr/lib64/python3.5/os.py", line 864, in fsencode
>> raise TypeError("expect bytes or str, not %s" % type(filename).__name__)
>> TypeError: expect bytes or str, not NoneType
>
> (This is progress! In 3.0.1, I didn't get any traceback at all.)
>
> The problem would appear to be in Grass7Utils.py (or be triggered there;
> perhaps my input is faulty). Here's the code around line 367 (first line
> is 359, last is 379)
>
>> with subprocess.Popen(
>> command,
>> shell=True if isMac() else False,
>> stdout=subprocess.PIPE,
>> stdin=subprocess.DEVNULL,
>> stderr=subprocess.STDOUT,
>> universal_newlines=True,
>> env=grassenv,
>> startupinfo=si if isWindows() else None
>> ) as proc:
>> for line in iter(proc.stdout.readline, ''):
>> if 'GRASS_INFO_PERCENT' in line:
>> try:
>> feedback.setProgress(int(line[len('GRASS_INFO_PERCENT') + 2:]))
>> except:
>> pass
>> else:
>> if 'r.out' in line or 'v.out' in line:
>> grassOutDone = True
>> loglines.append(line)
>> feedback.pushConsoleInfo(line)
>
> The offending statement in subprocess.py is this:
>
>> self._execute_child(args, executable, preexec_fn, close_fds,
>> pass_fds, cwd, env,
>> startupinfo, creationflags, shell,
>> p2cread, p2cwrite,
>> c2pread, c2pwrite,
>> errread, errwrite,
>> restore_signals, start_new_session)
>
> ('executable' is None in this case, which seems wrong, but I have no
> clue where it comes from.)
>
> The problem is not, as far as I can tell, in GRASS. The batch job gets
> generated and when I run it manually, it works fine:
>
>> /tmp/processing_b02dc280166f49e0af77208296099ea5/grassdata $ export GRASS_BATCH_JOB="$PWD/grass_batch_job.sh"
>> /tmp/processing_b02dc280166f49e0af77208296099ea5/grassdata $ grass72
>> Cleaning up temporary files...
>> Starting GRASS GIS...
>> Executing </tmp/processing_b02dc280166f49e0af77208296099ea5/grassdata/grass_batch_job.sh> ...
>> Projection information updated
>> Over-riding projection check
>> Check if OGR layer <Kenya_power_grid> contains polygons...
>> 100%
>> WARNING: Writing column <BANDWIDTH> with integer 64 as integer 32
>> WARNING: Writing column <AICD> with integer 64 as integer 32
>> Importing 48 features (OGR layer <Kenya_power_grid>)...
>> 100%
>> -----------------------------------------------------
>> Building topology for vector map <vector_5b086ca96575b3 at PERMANENT>...
>> Registering primitives...
>> 61 primitives registered
>> 180 vertices registered
>> Building areas...
>> 100%
>> 0 areas built
>> 0 isles built
>> Attaching islands...
>> Attaching centroids...
>> 100%
>> Number of nodes: 63
>> Number of primitives: 61
>> Number of points: 0
>> Number of lines: 61
>> Number of boundaries: 0
>> Number of centroids: 0
>> Number of areas: 0
>> Number of isles: 0
>> Writing raster map...
>> 100%
>> v.to.rast complete.
>> Checking GDAL data type and nodata value...
>> 100%
>> Using GDAL data type <Int32>
>> Input raster map contains cells with NULL-value (no-data). The value
>> -2147483648 will be used to represent no-data values in the input map. You
>> can specify a nodata value with the nodata option.
>> Exporting raster data to GTiff format...
>> ERROR 6: SetColorTable() only supported for Byte or UInt16 bands in TIFF format.
>> 100%
>> r.out.gdal complete. File
>> </tmp/processing_b02dc280166f49e0af77208296099ea5/5b086c8c419362> created.
>> Execution of </tmp/processing_b02dc280166f49e0af77208296099ea5/grassdata/grass_batch_job.sh> finished.
>> Cleaning up temporary files...
>
> What can I do to determine the cause and make this work?
>
> Cheers
>
> Stephen
>
More information about the Qgis-user
mailing list