[GRASS-user] new GRASS virtual raster in trunk

[Markus Metz]

On Mon, Jun 4, 2018 at 3:36 PM, Nikos Alexandris <nik at nikosalexandris.net>
wrote:
>
> * Rich Shepard <rshepard at appl-ecosys.com> [2018-06-04 05:56:16 -0700]:
>
>> On Mon, 4 Jun 2018, Markus Metz wrote:
>>
>>> A new GRASS virtual raster format has been added to trunk in r72761-4,
>>> together with a new module to build a GRASS VRT: r.buildvrt.
>>
>>
>> Markus,
>>
>>  For folks like me who have not followed this thread please explain the
>> problem for which a raster VRT is the solution.
>>
>> Best regards,
>>
>> Rich
>
>
> Dear Rich,
>
> (sorry Markus for my intervention -- just so excited about this new
> creation of yours!)
>
> if I may, virtual raster maps allow to treat an arbitrary number of
> raster maps as one map. Hence, it is not required to patch hundreds or
> thousands or more (?) raster map tiles, to build a single map.

Technically, there is no limit on the number of map tiles combined in a
VRT. However, reading should become a bit slower with more tiles,
independent of the size of the individual tiles.
>
> Why build a single map?  Because the one or the other module or workflow
> need it, in order to be able to process the entire computational extent
> of interest.
>
>
> In my case, I treat 177 tiles that cover European territories,
> for each of which there exists a raster map (of 10000 x 10000 pixels).
> Having one raster map, for the whole of Europe, will make it easier to
> extract zonal statistics, for example.
>
>
> So, in the general case of many raster maps, that form say, a "canonical"
> tile grid, zones of interest (say vector boundaries), for which is asked
> to extract statistics, are possibly split over two to up to four raster
> map tiles. It takes some respectable amount of time of handcrafting, to
> get stats for each tile, assembly then results for zones that don't
> entirely fall inside one tile.

In this case, using a huge single raster map should be slower, if only up
to four small tiles are needed. Here, a VRT is faster.

>
> I am sure there are better explanations or even more complicated
> problems for which a Virtual raster is the answer. In short, however,
> and the way I understand things, it is a "work-around" to avoid dealing
> with huge raster data sets. Huge as in file size. And, while the total
> processing time, might not necessarily drop down, depending on the
> work-flow, the time to script a solution decreases dramatically.

Thanks for the explanation Nikos!

There are more sophisticated use cases: you can distribute tiles over
different mapsets which may reside on different physical storage devices.
Simultaneous access to different parts of the VRT raster should now become
much faster. This can be useful for e.g. web processing services or also
parallel processing on a HPC.

Markus M

>
>
> Nikos, happy to be a GRASS GIS user.
>
> _______________________________________________
> grass-user mailing list
> grass-user at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-user
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20180604/d4c84ef4/attachment.html>