[GRASS-dev] Does multi-threading apply to r.series.lwr?

[Markus Metz]

On Fri, Nov 3, 2017 at 7:08 PM, Nikos Alexandris <nik at nikosalexandris.net>
wrote:
>
> Before processing tens of thousands of Landsat8 TIRS bands, a script
> tests the workflow described below, for 74 scenes, that correspond to one
> WRS2 tile.
>
>
> The script takes as inputs:
>
> <Landsat8Pool> path to directory with Landsat scenes
> <ScenePattern> regex pattern to match a set of Landsat scene identifiers
> <LandCover> land cover map
> <grassdb> path to the GRASS GIS data base
> <Location> name for the GRASS GIS Location
> <TargetMapset> name for the GRASS GIS Mapset to host maps to build a time
series
> <WindowSize> an odd integer, parameter for a split-window algorithm (SW)
>
>
> Part 1 of the workflow derives Land Surface Temperature maps by:
>
> 1. Creating the target Location
> 2. Linking pseudo GRASS raster maps to Landsat8 GeoTIFF files
> 3. Exporting a TGIS-compliant list of maps and timestamps
> 4. Importing a land cover map required for the SW algorithm
> 5. Estimating Land Surface Temperature (LST) maps for given scenes
(i.landsat8.swlst)
> 6. Creating a dedicated Mapset for LST maps
> 7. Copying LST maps in the "LST" Mapset
> 8. Removing initial LST maps from individual scene Mapsets
>
>
> With a somewhat strong CPU, producing one LST map (7771 rows by 7651
> columns = 59455921 cells), takes ~34 minutes.
>
> For 74 Landsat8 input scenes, first trials took about 42 hours, running
> grass-7.3.svn inside a docker container, albeit assigned one CPU.
>
>
> Part 2 concerns building Time Series by:
>
> 1. Creating and LST Spatio-Temporal Raster Dara Set (STRDS)
> 2. Registering LST maps in TGIS' data base
> 3. Smoothing the LST STRDS via Local Weighted Regression (r.series.lwr)
> 4. Timestamping Local-Weight-Regression derived maps
> 5. Creating an STRDS for LWR maps
> 6. Registering LWR maps in TGIS' data base
>
>
> Part 2 took about 130 minutes including all steps. Obviously, step 3 is
> practically the consumer.
>
>
> Overall it took about 44 hours to build an LWR Smoothed LST STRDS.
>
> After proofing the concept, good use of the cluster concerns steps 2, 5,
> 7 and 8 (of Part 1). Processes that can/should run in parallel, in an
> (admittedly heterogeneous) cluster that consists of 912 cores.
>
> Final concern is if multi-threading applies to r.series.lwr (step 3 of
> Part 2).

You could
1. create different temporal chunks of the time series, each chunk will be
processed by r.series.lwr, all chunks can be processed in parallel.
2. create different spatial chunks of the time series (tiling the
computational region), each chunk will be processed by r.series.lwr, all
chunks can be processed in parallel.

HTH,

Markus M
>
> Thank you, Nikos
>
> _______________________________________________
> grass-dev mailing list
> grass-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-dev/attachments/20171103/3ac4a68a/attachment.html>