[GRASS-dev] Fwd: Re: Adding an expert mode to the parser

[Moritz Lennert]

-------- Message d'origine --------
De : Moritz Lennert <moritzlennert at posteo.net>
Envoyé : 25 septembre 2016 23:21:04 GMT+02:00
À : "Sören Gebbert" <soerengebbert at googlemail.com>, Markus Neteler <neteler at osgeo.org>
Cc : GRASS developers list <grass-dev at lists.osgeo.org>, Markus Metz <markus.metz.giswork at gmail.com>
Objet : Re: [GRASS-dev] Adding an expert mode to the parser



Le 25 septembre 2016 22:29:19 GMT+02:00, "Sören Gebbert" <soerengebbert at googlemail.com> a écrit :
>Hi,
>Here an example of hidden region settings for modules utilized in the
>temporal framework:
>
>Imagine you have a Landsat location that contains many thousand scenes
>(time series) from all over the world including all bands. Each band of
>the
>time series is managed in a specific space-time dataset. You may want
>to
>use the temporal raster algebra to compute the NDVI in parallel on all
>scenes at once. This is not possible at the moment, because of the
>current
>region settings. Except you can set the current region to cover the
>whole
>world, which is not preferable.
>
>So the idea is to have options that will set the computational region
>for
>each process independently, when the process is invoked. Hence the
>raster
>computational window is set in the parser, so that the usual region get
>methods in the modules will work untouched. This avoids the current
>approach that is covering a single module call (like r.mapclalc,
>r.series,
>r.neighbors, ... ) in several g.region and g.remove calls in a
>dedicated
>subprocess.
>
>This option is only useful for parallel processing of raster layers
>with
>different spatial extents. A special feature for the temporal
>framework,
>that should not be exposed to all the users, only to those who
>implement
>massive parallel working temporal modules. Hence using the same
>approach as
>for WPS, GUI and HTML output:
>
>r.mapclac --raster-region= --north= --south= --west= --east= --res=
>--ewres= --nsres= --vect-region --raster-align= ...

Can't you do this at a higher level, i.e using the GRASS Python API's use_temp_region ?

Moritz


>
>Just my 2 cent
>Sören
>
>2016-09-25 20:49 GMT+01:00 Markus Neteler <neteler at osgeo.org>:
>
>> On Fri, Sep 23, 2016 at 11:30 PM, Markus Metz
>> <markus.metz.giswork at gmail.com> wrote:
>> > On Fri, Sep 23, 2016 at 11:22 PM, Markus Neteler
><neteler at osgeo.org>
>> wrote:
>> >> On Fri, Sep 23, 2016 at 11:05 PM, Markus Metz
>> >> <markus.metz.giswork at gmail.com> wrote:
>> >>> On Fri, Sep 23, 2016 at 1:11 PM, Markus Neteler
><neteler at osgeo.org>
>> wrote:
>> >> ...
>> >>> Your motivation is to provide a specialized CLI interface for HPC
>> >>> processing?
>> >>
>> >> No, not the case.
>> >>
>> >>> We used GRASS with HPC processing for years and the
>> >>> problems we faced were causes by the HPC hardware and software
>> >>> infrastructure, not by GRASS. What exactly is the problem with
>using
>> >>> GRASS and HPC processing?
>> >>
>> >> There is no problem. There is just the issue that with an
>increasing
>> >> amount of additions (e.g. maybe the need to provide
>region/resolution
>> >> to individual modules for independent parallel processing without
>the
>> >> overhead of always opening a new mapset)
>> >
>> > Getting closer it seems. Can you quantify "the overhead of always
>> > opening a new mapset"?
>>
>> As an example, when aiming at processing all Sentinel-2 tiles
>> globally, we speak about currently 73000 scenes * up-to-16 tiles
>along
>> with global data, analysis on top of other global data is more
>complex
>> when doing each job in its own mapset and reintegrate it in a single
>> target mapset as if able to process then in parallel in one mapset by
>> simply specifying the respective region to the command of interest.
>> Yes, different from the current paradigm and not for G7.
>>
>> But my original comment was targeted at the increasing number of
>> module parameters and how to handle that (with some new HPC related
>> idea as an example).
>>
>> I'm fine to archive this question for now, it will likely come up
>> again in the future.
>>
>> markusN
>> _______________________________________________
>> grass-dev mailing list
>> grass-dev at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/grass-dev
>>
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>grass-dev mailing list
>grass-dev at lists.osgeo.org
>http://lists.osgeo.org/mailman/listinfo/grass-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-dev/attachments/20160925/6c6bab7c/attachment-0001.html>