[GRASS-user] [GRASS-dev] new GRASS virtual raster in trunk

[Moritz Lennert]

Am 4. Juni 2018 20:17:58 MESZ schrieb Markus Neteler <neteler at osgeo.org>:
>(cc grass-dev for the record)
>
>On Mon, Jun 4, 2018 at 4:59 PM, Markus Metz
><markus.metz.giswork at gmail.com> wrote:
>> On Mon, Jun 4, 2018 at 3:36 PM, Nikos Alexandris
><nik at nikosalexandris.net>
>> wrote:
>...
>>>> On Mon, 4 Jun 2018, Markus Metz wrote:
>>>>> A new GRASS virtual raster format has been added to trunk in
>r72761-4,
>>>>> together with a new module to build a GRASS VRT: r.buildvrt.
>>>>
>...
>>> if I may, virtual raster maps allow to treat an arbitrary number of
>>> raster maps as one map. Hence, it is not required to patch hundreds
>or
>>> thousands or more (?) raster map tiles, to build a single map.
>>
>> Technically, there is no limit on the number of map tiles combined in
>a VRT.
>> However, reading should become a bit slower with more tiles,
>independent of
>> the size of the individual tiles.
>>>
>>> Why build a single map?  Because the one or the other module or
>workflow
>>> need it, in order to be able to process the entire computational
>extent
>>> of interest.
>>>
>>>
>>> In my case, I treat 177 tiles that cover European territories,
>>> for each of which there exists a raster map (of 10000 x 10000
>pixels).
>>> Having one raster map, for the whole of Europe, will make it easier
>to
>>> extract zonal statistics, for example.
>>>
>>>
>>> So, in the general case of many raster maps, that form say, a
>"canonical"
>>> tile grid, zones of interest (say vector boundaries), for which is
>asked
>>> to extract statistics, are possibly split over two to up to four
>raster
>>> map tiles. It takes some respectable amount of time of handcrafting,
>to
>>> get stats for each tile, assembly then results for zones that don't
>>> entirely fall inside one tile.
>>
>> In this case, using a huge single raster map should be slower, if
>only up to
>> four small tiles are needed. Here, a VRT is faster.
>>
>>>
>>> I am sure there are better explanations or even more complicated
>>> problems for which a Virtual raster is the answer. In short,
>however,
>>> and the way I understand things, it is a "work-around" to avoid
>dealing
>>> with huge raster data sets. Huge as in file size. And, while the
>total
>>> processing time, might not necessarily drop down, depending on the
>>> work-flow, the time to script a solution decreases dramatically.
>>
>> Thanks for the explanation Nikos!
>>
>> There are more sophisticated use cases: you can distribute tiles over
>> different mapsets which may reside on different physical storage
>devices.
>> Simultaneous access to different parts of the VRT raster should now
>become
>> much faster. This can be useful for e.g. web processing services or
>also
>> parallel processing on a HPC.
>>
>> Markus M
>
>So cool, thanks Markus!!

Yes, huge kudos to Markus for this new feature. As we've seen on the list and elsewhere interest for using GRASS in HPC environments is already high and still growing, and this is a tremendous enhancement !

Moritz


>
>markusN
>_______________________________________________
>grass-dev mailing list
>grass-dev at lists.osgeo.org
>https://lists.osgeo.org/mailman/listinfo/grass-dev