[GRASS-dev] [GRASS-user] new GRASS virtual raster in trunk

[Markus Neteler]

(cc grass-dev for the record)

On Mon, Jun 4, 2018 at 4:59 PM, Markus Metz
<markus.metz.giswork at gmail.com> wrote:
> On Mon, Jun 4, 2018 at 3:36 PM, Nikos Alexandris <nik at nikosalexandris.net>
> wrote:
...
>>> On Mon, 4 Jun 2018, Markus Metz wrote:
>>>> A new GRASS virtual raster format has been added to trunk in r72761-4,
>>>> together with a new module to build a GRASS VRT: r.buildvrt.
>>>
...
>> if I may, virtual raster maps allow to treat an arbitrary number of
>> raster maps as one map. Hence, it is not required to patch hundreds or
>> thousands or more (?) raster map tiles, to build a single map.
>
> Technically, there is no limit on the number of map tiles combined in a VRT.
> However, reading should become a bit slower with more tiles, independent of
> the size of the individual tiles.
>>
>> Why build a single map?  Because the one or the other module or workflow
>> need it, in order to be able to process the entire computational extent
>> of interest.
>>
>>
>> In my case, I treat 177 tiles that cover European territories,
>> for each of which there exists a raster map (of 10000 x 10000 pixels).
>> Having one raster map, for the whole of Europe, will make it easier to
>> extract zonal statistics, for example.
>>
>>
>> So, in the general case of many raster maps, that form say, a "canonical"
>> tile grid, zones of interest (say vector boundaries), for which is asked
>> to extract statistics, are possibly split over two to up to four raster
>> map tiles. It takes some respectable amount of time of handcrafting, to
>> get stats for each tile, assembly then results for zones that don't
>> entirely fall inside one tile.
>
> In this case, using a huge single raster map should be slower, if only up to
> four small tiles are needed. Here, a VRT is faster.
>
>>
>> I am sure there are better explanations or even more complicated
>> problems for which a Virtual raster is the answer. In short, however,
>> and the way I understand things, it is a "work-around" to avoid dealing
>> with huge raster data sets. Huge as in file size. And, while the total
>> processing time, might not necessarily drop down, depending on the
>> work-flow, the time to script a solution decreases dramatically.
>
> Thanks for the explanation Nikos!
>
> There are more sophisticated use cases: you can distribute tiles over
> different mapsets which may reside on different physical storage devices.
> Simultaneous access to different parts of the VRT raster should now become
> much faster. This can be useful for e.g. web processing services or also
> parallel processing on a HPC.
>
> Markus M

So cool, thanks Markus!!

markusN