[GRASS-dev] ParallelModuleQueue freezes...

Sun May 30 12:27:05 PDT 2021

Hi Anna,

Thanks for clarification.
Would be great if you could backport #1407. I don`t know if the PR depends on others...
But if it is safe to backport just this one, I could do that too of course.

In core, MultiModule is used just in t.rast.neighbors, it seems. And that module thus does not work in GRASS 7.8 right now. So the current state can be considered a bug...

Would be great if it would be possible to get that into 7.8.6 somehow, even if there is already a RC. I am about to finish a new AddOn, where I would like to use the combination of MultiModule and ParallelModuleQueue. The development version of GRASS will not be available in the production environment in my institute for quite some time, so I would have to wait for some month to use the addon...

Cheers,
Stefan

From: Anna Petrášová <kratochanna at gmail.com>
Sent: fredag 28. mai 2021 16:36
To: Stefan Blumentrath <Stefan.Blumentrath at nina.no>
Cc: Luca Delucchi <lucadeluge at gmail.com>; GRASS developers list <grass-dev at lists.osgeo.org>
Subject: Re: [GRASS-dev] ParallelModuleQueue freezes...

On Fri, May 28, 2021 at 10:02 AM Stefan Blumentrath <Stefan.Blumentrath at nina.no<mailto:Stefan.Blumentrath at nina.no>> wrote:
Some more context.
The t.rast.neighbors test seems to work with the current dev version
"2048e1016" from 2021-05-19T09:38:49+00:00
It fails however, with 7.8.5 from 2020-12-22T15:55:09+00:00 and an older development version "35edbf7dc"
from 2020-06-03T16:03:52+00:00

So, maybe one or more of the recent changes need backporting?
https://github.com/OSGeo/grass/commits/master/python/grass/pygrass/modules/interface/module.py<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOSGeo%2Fgrass%2Fcommits%2Fmaster%2Fpython%2Fgrass%2Fpygrass%2Fmodules%2Finterface%2Fmodule.py&data=04%7C01%7C%7C8fb6cc6211854d8631f708d921e5fa6e%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637578093904701621%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=HYHd7YUo5l9wQD%2Fdooe3zC%2BYCd%2BL%2B3WRMbX5OeCjI5U%3D&reserved=0>

This one:
https://github.com/OSGeo/grass/pull/1407<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOSGeo%2Fgrass%2Fpull%2F1407&data=04%7C01%7C%7C8fb6cc6211854d8631f708d921e5fa6e%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637578093904711618%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=SO3%2FOC9wbs1WXlumRUJYpqn0r5i3LfMcGxk7STcpPII%3D&reserved=0>

The reason I didn't backport it was there has been a small change in API to make it work. But Python3 support is probably a good reason to backport it.

In the 7_8 release branch, the latest change is from May 2019:
https://github.com/OSGeo/grass/commits/releasebranch_7_8/lib/python/pygrass/modules/interface/module.py<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOSGeo%2Fgrass%2Fcommits%2Freleasebranch_7_8%2Flib%2Fpython%2Fpygrass%2Fmodules%2Finterface%2Fmodule.py&data=04%7C01%7C%7C8fb6cc6211854d8631f708d921e5fa6e%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637578093904711618%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=pm4%2BxQc8G9FSlaad%2FsXEBzKwhs4lxC%2BXSvRWLRqsjWg%3D&reserved=0>

Hope that helps identifying a solution?

Cheers
Stefan

-----Original Message-----
From: Stefan Blumentrath <Stefan.Blumentrath at nina.no<mailto:Stefan.Blumentrath at nina.no>>
Sent: torsdag 27. mai 2021 23:09
To: Stefan Blumentrath <Stefan.Blumentrath at nina.no<mailto:Stefan.Blumentrath at nina.no>>; Luca Delucchi <lucadeluge at gmail.com<mailto:lucadeluge at gmail.com>>; Anna Petrášová <kratochanna at gmail.com<mailto:kratochanna at gmail.com>>
Cc: GRASS developers list <grass-dev at lists.osgeo.org<mailto:grass-dev at lists.osgeo.org>>
Subject: RE: [GRASS-dev] ParallelModuleQueue freezes...

See:
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOSGeo%2Fgrass%2Fpull%2F1600&data=04%7C01%7C%7Cc5fe909715a647431c9308d92153b445%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637577465653048701%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=5yfxG5gjlYlzgJn6yTP9zbUk7GE1JdcYPxWIQFwUK%2Fo%3D&reserved=0<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOSGeo%2Fgrass%2Fpull%2F1600&data=04%7C01%7C%7C8fb6cc6211854d8631f708d921e5fa6e%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637578093904721611%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Kw1pcn3mIlEK8hRS70B966zqhgpud9cB3bcKA2KmRwI%3D&reserved=0>
Please test.
Unittests for t.rast.neighbors should work again with this small change...

-----Original Message-----
From: grass-dev <grass-dev-bounces at lists.osgeo.org<mailto:grass-dev-bounces at lists.osgeo.org>> On Behalf Of Stefan Blumentrath
Sent: torsdag 27. mai 2021 08:19
To: Luca Delucchi <lucadeluge at gmail.com<mailto:lucadeluge at gmail.com>>; Anna Petrášová <kratochanna at gmail.com<mailto:kratochanna at gmail.com>>
Cc: GRASS developers list <grass-dev at lists.osgeo.org<mailto:grass-dev at lists.osgeo.org>>
Subject: Re: [GRASS-dev] ParallelModuleQueue freezes...

The combination ParallelModuleQueue and MultiModule is used in t.rast.neighbors: https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOSGeo%2Fgrass%2Fblob%2Fmaster%2Ftemporal%2Ft.rast.neighbors%2Ft.rast.neighbors.py&data=04%7C01%7C%7Cc5fe909715a647431c9308d92153b445%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637577465653048701%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=5QyN%2FR1cXgoWWGS2lgjKi8%2BeLX77w37bfMQzcP6O4bM%3D&reserved=0<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOSGeo%2Fgrass%2Fblob%2Fmaster%2Ftemporal%2Ft.rast.neighbors%2Ft.rast.neighbors.py&data=04%7C01%7C%7C8fb6cc6211854d8631f708d921e5fa6e%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637578093904721611%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=9m2VxOc8E%2B3VYEO%2BUo76zmi8dJP%2By1aJOplRlcQHE9w%3D&reserved=0>

The testsuite of that module is deactivated for Python > 2.
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOSGeo%2Fgrass%2Fblob%2Fmaster%2Ftemporal%2Ft.rast.neighbors%2Ftestsuite%2Ftest_neighbors.py&data=04%7C01%7C%7Cc5fe909715a647431c9308d92153b445%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637577465653048701%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=nQTdDIGmefI6F37zdjhyMDBYUrIxVaC%2FiA5FzeUK2KU%3D&reserved=0<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOSGeo%2Fgrass%2Fblob%2Fmaster%2Ftemporal%2Ft.rast.neighbors%2Ftestsuite%2Ftest_neighbors.py&data=04%7C01%7C%7C8fb6cc6211854d8631f708d921e5fa6e%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637578093904731607%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=%2Fp6TEIL8K5hk%2B2GXqK4eYRJohbS0urzuwqqHAW0p5v0%3D&reserved=0>

If you activate it by commenting out line 15. You will see the behavior...

Thanks for looking into this!

Cheers
Stefan

-----Original Message-----
From: Luca Delucchi <lucadeluge at gmail.com<mailto:lucadeluge at gmail.com>>
Sent: torsdag 27. mai 2021 06:07
To: Anna Petrášová <kratochanna at gmail.com<mailto:kratochanna at gmail.com>>
Cc: Stefan Blumentrath <Stefan.Blumentrath at nina.no<mailto:Stefan.Blumentrath at nina.no>>; GRASS developers list <grass-dev at lists.osgeo.org<mailto:grass-dev at lists.osgeo.org>>
Subject: Re: [GRASS-dev] ParallelModuleQueue freezes...

On Thu, 27 May 2021 at 04:07, Anna Petrášová <kratochanna at gmail.com<mailto:kratochanna at gmail.com>> wrote:
>
> Could you provide an example that is failing? The doctests are running for me. I spent some time fixing it a couple months ago. I use Python 3.6, I wonder if newer Python would cause some issues.
>

https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOSGeo%2Fgrass-addons%2Fpull%2F523%2Ffiles&data=04%7C01%7C%7Cc5fe909715a647431c9308d92153b445%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637577465653048701%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=KT5h55J89LLXhK7uEujAmIwfksjPGyXEy2u7gYIc2Ns%3D&reserved=0<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOSGeo%2Fgrass-addons%2Fpull%2F523%2Ffiles&data=04%7C01%7C%7C8fb6cc6211854d8631f708d921e5fa6e%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637578093904731607%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=E599W%2Fe7tP3FeODqu5U74aoUlR3miVrIev5CmefwNrg%3D&reserved=0> line 544, the commented lines are not working for me..
If you want to try t.vi<https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Ft.vi%2F&data=04%7C01%7C%7C8fb6cc6211854d8631f708d921e5fa6e%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637578093904741601%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=tYSyPA80Tdemea1q%2BI9vdviTm%2BawnMOADfQaton3o0A%3D&reserved=0> just comment line 543 and uncomment the following lines..

> Anna
>

--
ciao
Luca

https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.lucadelu.org%2F&data=04%7C01%7C%7Cc5fe909715a647431c9308d92153b445%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637577465653048701%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=hM8CpSgjw7SFjGKxM9SFfWI3G8U3nr8eeCqedZORdso%3D&reserved=0<https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.lucadelu.org%2F&data=04%7C01%7C%7C8fb6cc6211854d8631f708d921e5fa6e%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637578093904751599%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=QoJKaET2VizmUlzygcEl3cpapqheK1Y8yJUbVIP%2Fupk%3D&reserved=0>
_______________________________________________
grass-dev mailing list
grass-dev at lists.osgeo.org<mailto:grass-dev at lists.osgeo.org>
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fgrass-dev&data=04%7C01%7C%7Cc5fe909715a647431c9308d92153b445%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637577465653048701%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=IwP5QxiLzELTGrr393lqAWrlWRhBVg%2FbOhrz9lBHMkw%3D&reserved=0<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fgrass-dev&data=04%7C01%7C%7C8fb6cc6211854d8631f708d921e5fa6e%7C6cef373021314901831055b3abf02c73%7C0%7C0%7C637578093904751599%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=VTqN8LRYcDOnA3VS%2B%2B3lKKvG6A2XDDp%2Bdbusc407V2c%3D&reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-dev/attachments/20210530/6b7c985d/attachment-0001.html>